An intelligent database metadata management system

Abstract

The present invention provides an intelligent database metadata management system, which relates to the technical field of data management. The system includes: acquiring the metadata of a database and performing data analysis to determine the data correlation between the metadata; determining the data transmission source of the metadata and classifying the metadata, constructing a metadata management platform, determining the distributed architecture of the metadata, and performing metadata management according to the data correlation between the metadata and the load of the metadata management platform; performing storage management on the metadata according to the metadata management result; realizing intelligent database metadata management.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to an intelligent database metadata management system. Background Art

[0002] With the development of society, the stored data in the database is becoming more and more numerous, and data management faces many challenges; firstly, the efficiency of users calling relevant data from the database when accessing data is low, and it is difficult to link relevant data; in addition, a large amount of stored data makes the database load seriously, and also leads to the risk of data leakage in the data storage of the database.

[0003] Therefore, the present invention provides an intelligent database metadata management system. Summary of the Invention

[0004] The present invention provides an intelligent database metadata management system, which determines the data correlation between metadata by obtaining the metadata of the database and performing data analysis; determines the data transmission source of the metadata and classifies the metadata, constructs a metadata management platform, determines the distributed architecture of the metadata, and performs metadata management according to the data correlation between the metadata and the load of the metadata management platform; performs storage management on the metadata according to the metadata management result; realizes the intelligent management of database metadata.

[0005] The present invention provides an intelligent database metadata management system, including:

[0006] A metadata association module, configured to obtain the metadata of the database and perform data analysis to determine the data correlation between the metadata;

[0007] A metadata management module, configured to determine the data transmission source of the metadata and classify the metadata, construct a metadata management platform, determine the distributed architecture of the metadata, and perform metadata management according to the data correlation between the metadata and the load of the metadata management platform;

[0008] A metadata storage module, configured to perform storage management on the metadata according to the metadata management result.

[0009] According to an intelligent database metadata management system provided by the present invention, the metadata association module includes:

[0010] A metadata acquisition unit, configured to automatically acquire the stored data of the database and obtain the metadata of the stored data according to a preset data extraction strategy;

[0011] A metadata classification unit, configured to perform a primary metadata classification process on the metadata of the stored data according to the data type of the stored data, and classify it into text type metadata, image type metadata, and other type metadata;

[0012] Classify the metadata of the stored data into two-dimensional data according to the metadata type, and divide it into inherent class metadata, management class metadata, and description class metadata.

[0013] According to an intelligent database metadata management system provided by the present invention, a metadata association module includes:

[0014] A metadata association unit for determining that there is a metadata association between stored data when there is a data link between the stored data;

[0015] An association refinement unit for obtaining the metadata corresponding to the stored data with a metadata association, denoted as associated metadata;

[0016] Perform an association refinement analysis on the associated metadata according to the one-dimensional metadata classification processing result and the two-dimensional data classification processing result. When there is a data link between the associated metadata in a metadata classification processing result, determine that the metadata type corresponding to a metadata classification processing result is the association refinement type of the associated metadata;

[0017] When there is no data link between the associated metadata in a metadata classification processing result, determine that the metadata type corresponding to a metadata classification processing result is the association irrelevant type of the associated metadata;

[0018] A data correlation unit for extracting all the stored data corresponding to the association refinement type of the associated metadata for data correlation analysis to determine the data correlation between the metadata.

[0019] According to an intelligent database metadata management system provided by the present invention, a metadata management module includes:

[0020] A transmission source metadata classification unit for obtaining the data transmission relationship between metadata and determining the data transmission source of the metadata;

[0021] Perform three-dimensional metadata classification processing according to the data transmission source of the metadata;

[0022] A classification management unit for adding classification management labels to the metadata according to the one-dimensional metadata classification processing result, the two-dimensional data classification processing result, and the three-dimensional metadata classification processing result for classification management, where the classification management labels include: one-dimensional metadata classification management label, two-dimensional data classification management label, and three-dimensional metadata classification management label.

[0023] According to an intelligent database metadata management system provided by the present invention, a metadata management module includes:

[0024] A platform construction unit, which is used to obtain metadata management requirements, construct a metadata management platform, and determine the distributed architecture of metadata;

[0025] A distributed storage management unit, which is used to obtain the data storage space of metadata according to the distributed architecture of metadata, divide the data storage space area according to the number of classifications of the metadata classification processing results, and allocate the data storage space amount in combination with the amount of metadata according to the number of classification management tags of metadata, and perform distributed storage management on metadata.

[0026] According to an intelligent database metadata management system provided by the present invention, a metadata management module includes:

[0027] A load management unit, which is used to allocate the load amount of the metadata management platform according to the distributed storage management result, and obtain the load allocation amount of metadata;

[0028] Randomly calibrate a piece of metadata storing data as standard metadata, and calculate the load utilization rate of the metadata management platform according to the data correlation between other metadata and the standard metadata and the load allocation amount of the standard metadata;

[0029] ; wherein, represents the load utilization rate of the metadata management platform; represents the load allocation amount of the standard metadata; represents the th occurrence times of the th other metadata in the corresponding stored data; represents the data quantity of the corresponding stored data of the th other metadata; represents the th occurrence times of the th other metadata in the metadata management platform; represents the th data correlation between the th other metadata and the standard metadata; represents the th data quantity of the th other metadata in the metadata management platform; represents the data quantity of the standard metadata in the metadata management platform; represents the load amount of the metadata management platform; represents the number of other metadata;

[0030] When the load utilization rate of the metadata management platform is not lower than the preset utilization rate, it is determined that the metadata load management is completed;

[0031] A load migration management unit, which is used to determine the individual load utilization rate of metadata according to the load allocation amount of metadata and the load usage amount of metadata when the load utilization rate of the metadata management platform is lower than the preset utilization rate;

[0032] Sort the metadata according to the individual load utilization rate of the metadata, and mark the metadata with an individual load utilization rate not higher than the load utilization rate of the metadata management platform as metadata with insufficient load utilization;

[0033] Determine the migratable load allocation amount of the metadata with insufficient load utilization according to the individual load utilization rate of the metadata, the load utilization rate of the metadata management platform, and the load allocation amount of the metadata;

[0034] Obtain the migratable load allocation metadata of the metadata with insufficient load utilization according to the data correlation between the metadata, and re-allocate the load amount of the metadata management platform in combination with the migratable load allocation amount of the metadata with insufficient load utilization.

[0035] According to an intelligent database metadata management system provided by the present invention, the metadata storage module includes:

[0036] The metadata hierarchical storage unit is used to determine the storage level of the metadata according to the load allocation amount of the metadata, and perform hierarchical storage management on the metadata. Among them, the storage levels of the metadata include: hot metadata storage level, general metadata storage level, and cold metadata storage level;

[0037] The metadata clearing unit is used to set the metadata storage management period, regularly update the storage level of the metadata, and perform metadata clearing management on the metadata at the cold metadata storage level according to the storage time.

[0038] According to an intelligent database metadata management system provided by the present invention, it further includes:

[0039] The metadata visualization module is used to visually display the metadata management result.

[0040] Compared with the prior art, the beneficial effects of the present application are as follows:

[0041] By obtaining the metadata of the database and performing data analysis, determine the data correlation between the metadata; determine the data transmission source of the metadata and classify the metadata, construct a metadata management platform, determine the distributed architecture of the metadata, and perform metadata management according to the data correlation between the metadata and the load amount of the metadata management platform; perform storage management on the metadata according to the metadata management result; realize intelligent database metadata management.

[0042] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained by the structures specifically pointed out in the written specification and the drawings.

[0043] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Description of the Drawings

[0044] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, but do not constitute a limitation to the present invention. In the accompanying drawings:

[0045] Figure 1 It is a schematic structural diagram of an intelligent database metadata management system provided by an embodiment of the present invention. Detailed Embodiments

[0046] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0047] Embodiment 1:

[0048] An embodiment of the present invention provides an intelligent database metadata management system, as Figure 1 shown, including:

[0049] A metadata association module, configured to obtain the metadata of the database and perform data analysis to determine the data correlation between the metadata;

[0050] A metadata management module, configured to determine the data transmission source of the metadata and perform metadata classification, construct a metadata management platform, determine the distributed architecture of the metadata, and perform metadata management according to the data correlation between the metadata and the load of the metadata management platform;

[0051] A metadata storage module, configured to perform storage management on the metadata according to the metadata management result.

[0052] In this embodiment, the metadata of the database refers to the data used to describe the stored data of the database, that is, the smallest data unit, used to represent the stored data. For example, the data type and data storage location of the stored data a1 can be regarded as the metadata of the stored data a1.

[0053] In this embodiment, the data correlation between the metadata refers to the association relationship between the metadata determined according to the data analysis of the metadata. For example, if metadata b1 and metadata b2 belong to the same storage area, it is determined that there is a data correlation between metadata b1 and metadata b2.

[0054] In this embodiment, the data transmission source of metadata refers to the data transmission call source when accessing metadata, and is the data source determined according to the data transmission relationship between metadata. For example, the data transmission source a1 of metadata b1, that is, metadata b1 is obtained from the data transmission source a1.

[0055] In this embodiment, the data transmission source of metadata is determined and metadata is classified. The metadata classification result is determined according to the data transmission source. For example, metadata is classified according to the data transmission source a1 for subsequent metadata management.

[0056] In this embodiment, the metadata management platform refers to a platform for intelligent management of metadata.

[0057] In this embodiment, the distributed architecture of metadata refers to the architecture determined according to the distributed storage of metadata. Due to numerous databases and different types of metadata obtained, a distributed architecture is adopted.

[0058] In this embodiment, the load of the metadata management platform refers to the total load distribution for the metadata management platform to achieve metadata management.

[0059] In this embodiment, the metadata management result refers to the result of management based on the data correlation between metadata and the load of the metadata management platform.

[0060] In this embodiment, storage management of metadata is performed according to the metadata management result. For example, hierarchical storage management of metadata is performed to reasonably utilize the load of the metadata management platform; cleaning management of metadata is performed to avoid a large amount of historical data in the metadata management platform, thereby affecting the metadata management efficiency.

[0061] The working principle and beneficial effects of the above technical solution are as follows: Obtain the metadata of the database and perform data analysis to determine the data correlation between metadata; determine the data transmission source of metadata and classify metadata, construct a metadata management platform, determine the distributed architecture of metadata, and perform metadata management according to the data correlation between metadata and the load of the metadata management platform; perform storage management of metadata according to the metadata management result; realize intelligent database metadata management.

[0062] Embodiment 2:

[0063] The embodiment of the present invention provides an intelligent database metadata management system, a metadata association module, including:

[0064] A metadata acquisition unit for automatically acquiring the stored data of the database and obtaining the metadata of the stored data according to a preset data extraction strategy;

[0065] A metadata classification unit is used to perform a primary metadata classification process on the metadata of stored data according to the data type of the stored data, and classify it into text - type metadata, image - type metadata, and other - type metadata;

[0066] Perform a secondary data classification process on the metadata of stored data according to the metadata type, and classify it into inherent - type metadata, management - type metadata, and description - type metadata.

[0067] In this embodiment, the preset data extraction strategy refers to the strategy used to extract metadata by obtaining stored data. For example, it is the strategy of extracting keywords of stored data.

[0068] In this embodiment, the primary metadata classification process refers to the classification process performed according to the data type of the stored data. Among them, the data types of the stored data include text data type, image data type, and other data types.

[0069] In this embodiment, the secondary data classification process refers to the classification process performed according to the metadata type. Among them, the metadata types include: inherent - type metadata type, management - type metadata type, and description - type metadata type.

[0070] In this embodiment, the inherent - type metadata refers to the structure that describes the data and is used to identify storage, etc. For example, the storage address of the data, the database name corresponding to the data; the management - type metadata refers to the management attributes that describe the data. For example, the stored data belongs to database a1; the description - type metadata refers to the content that describes the data and is used to represent the stored data. For example, the keywords of the stored data.

[0071] The working principle and beneficial effects of the above - mentioned technical solution are: By obtaining the metadata of the database, performing the primary metadata classification process and the secondary data classification process, it lays a data classification foundation for subsequent metadata management.

[0072] Embodiment 3:

[0073] The embodiment of the present invention provides an intelligent database metadata management system, and the metadata association module includes:

[0074] A metadata association unit is used to determine that there is a metadata association between stored data when there is a data link between the stored data;

[0075] An association refinement unit is used to obtain the metadata corresponding to the stored data with metadata association, denoted as associated metadata;

[0076] Perform an associated refinement analysis on the associated metadata according to the one-dimensional metadata classification processing result and the two-dimensional metadata classification processing result. When there is a data link for the associated metadata in a metadata classification processing result, determine that the metadata type corresponding to a metadata classification processing result is the associated refinement type of the associated metadata;

[0077] When there is no data link for the associated metadata in a metadata classification processing result, determine that the metadata type corresponding to a metadata classification processing result is the associated irrelevant type of the associated metadata;

[0078] A data correlation unit is used to extract all stored data corresponding to the associated refinement type of the associated metadata for data correlation analysis to determine the data correlation between metadata.

[0079] In this embodiment, when there is a data link between stored data, the data link refers to a reference relationship between stored data, etc. For example, if stored data a1 references stored data a2, it is determined that there is a data link between stored data a1 and stored data a2.

[0080] In this embodiment, metadata association means that there is an association between the metadata of stored data with a data link. For example, there is a metadata association between metadata b1 of stored data a1 and metadata b2 of stored data a2.

[0081] In this embodiment, associated metadata refers to the metadata corresponding to stored data with a metadata association. For example, associated metadata (b1, b2).

[0082] In this embodiment, associated refinement analysis means determining the specific association type of metadata and then performing a specific association analysis. For example, perform an associated refinement analysis on the associated metadata according to the one-dimensional metadata classification processing result and the two-dimensional metadata classification processing result. If the associated metadata (b1, b2) both belong to the inherent class metadata, further perform an association analysis on the associated metadata according to the inherent class metadata.

[0083] In this embodiment, a metadata classification processing result refers to one of the one-dimensional metadata classification processing result and the two-dimensional metadata classification processing result.

[0084] In this embodiment, the associated refinement type refers to the metadata type with a data link determined by performing an associated refinement analysis on the associated metadata according to the one-dimensional metadata classification processing result and the two-dimensional metadata classification processing result. For example, the associated refinement type of the associated metadata (b1, b2) is the inherent class metadata type.

[0085] In this embodiment, the associated irrelevant type refers to the metadata type without data links determined by performing an associated refinement analysis on the associated metadata based on the classification processing results of one-dimensional metadata and the classification processing results of two-dimensional data. For example, the associated refinement type of the associated metadata (b1, b2) is the associated metadata type.

[0086] The working principle and beneficial effects of the above technical solution are as follows: By determining the metadata associations between stored data, performing associated refinement, and conducting data correlation analysis based on the associated refinement type, the data correlation between metadata is determined, ensuring the management quality of subsequent metadata management.

[0087] Embodiment 4:

[0088] An embodiment of the present invention provides an intelligent database metadata management system, and a metadata management module, including:

[0089] A transmission source metadata classification unit, configured to obtain the data transmission relationship between metadata and determine the data transmission source of the metadata;

[0090] Perform three-dimensional metadata classification processing according to the data transmission source of the metadata;

[0091] A classification management unit, configured to add classification management labels to the metadata for classification management based on the classification processing results of one-dimensional metadata, the classification processing results of two-dimensional data, and the classification processing results of three-dimensional data, where the classification management labels include: one-dimensional metadata classification management labels, two-dimensional data classification management labels, and three-dimensional data classification management labels.

[0092] In this embodiment, the data transmission relationship between metadata, for example, metadata a1 is obtained from metadata a2, and metadata a2 belongs to database b1, then there is a data transmission relationship between metadata a1 and metadata a2.

[0093] In this embodiment, to obtain the data transmission relationship between metadata and determine the data transmission source of the metadata, for example, the data transmission source of metadata a1 is database b1.

[0094] In this embodiment, the three-dimensional metadata classification processing refers to the classification processing performed according to the data transmission source of the metadata, which is used to determine the data transmission source and further provide a data classification basis for subsequent data management.

[0095] In this embodiment, the classification management label refers to the label determined according to the metadata classification processing results and is used to represent the metadata.

[0096] The working principle and beneficial effects of the above technical solution are as follows: By obtaining the data transmission relationships between metadata, determining the data transmission sources of the metadata, and performing three - dimensional metadata classification processing, the management quality of subsequent metadata management is further ensured.

[0097] Embodiment 5:

[0098] An embodiment of the present invention provides an intelligent database metadata management system. The metadata management module includes:

[0099] A platform construction unit, which is used to obtain metadata management requirements, construct a metadata management platform, and determine the distributed architecture of the metadata.

[0100] A distributed storage management unit, which is used to obtain the data storage space of the metadata according to the distributed architecture of the metadata, divide the data storage space area according to the classification quantity of the metadata classification processing result, and allocate the data storage space quantity according to the combination of the classification management label quantity of the metadata and the data volume of the metadata, so as to perform distributed storage management on the metadata.

[0101] In this embodiment, the metadata management requirements include, for example, data security requirements and data processing efficiency requirements.

[0102] In this embodiment, the data storage space of the metadata is obtained according to the distributed architecture of the metadata. The more complex the distributed architecture of the metadata is, the larger the data storage space of the metadata is. The complexity of the distributed architecture of the metadata is determined according to the number of sub - modules included in the distributed architecture of the metadata. For example, the distributed architecture of the metadata includes a1 number of sub - modules, corresponding to the complexity b1 of the distributed architecture of the metadata. If an average of 1 sub - module is allocated a data storage space c1 for the metadata, then the data storage space of the metadata obtained according to the distributed architecture of the metadata is a1 * c1.

[0103] In this embodiment, the data storage space area is divided according to the classification quantity of the metadata classification processing result. For example, if the classification quantity of the metadata classification processing result is d1, then the data storage space is divided into d1 number of data storage space areas for subsequent management of various metadata classification processing results.

[0104] In this embodiment, the data storage space quantity is allocated according to the combination of the classification management label quantity of the metadata and the data volume of the metadata. For example, a data storage space quantity is calibrated. The more classification management label data of the metadata, the more data storage space quantity is allocated; the more data volume of the metadata, the more data storage space quantity is allocated.

[0105] In this embodiment, the distributed storage management of metadata refers to the management carried out by dividing the data storage space area according to the number of classifications of the metadata classification processing results and allocating the data storage space amount in combination with the number of classification management tags of the metadata and the data volume of the metadata.

[0106] The working principle and beneficial effects of the above technical solution are: by constructing a metadata management platform, determining the distributed architecture of the metadata, and performing distributed storage management on the metadata, the storage quality of the metadata is ensured.

[0107] Embodiment 6:

[0108] The embodiment of the present invention provides an intelligent database metadata management system, and the metadata management module includes:

[0109] A load management unit, configured to allocate the load amount of the metadata management platform according to the distributed storage management result and obtain the load allocation amount of the metadata;

[0110] Randomly calibrate a metadata storing data as the standard metadata, and calculate the load utilization rate of the metadata management platform according to the data correlation between other metadata and the standard metadata and the load allocation amount of the standard metadata;

[0111] ; where, represents the load utilization rate of the metadata management platform; represents the load allocation amount of the standard metadata; represents the th occurrence times of the th other metadata in the corresponding stored data; represents the data volume of the corresponding stored data of the th other metadata; represents the occurrence times of the th other metadata in the metadata management platform; represents the data correlation between the th other metadata and the standard metadata; represents the data volume of the th other metadata in the metadata management platform; represents the data volume of the standard metadata in the metadata management platform; represents the load amount of the metadata management platform;

[0112] When the load utilization rate of the metadata management platform is not lower than the preset utilization rate, it is determined that the metadata load management is completed;

[0113] A load migration management unit, which is used to determine the individual load utilization rate of metadata according to the load allocation amount and the load usage amount of metadata when the load utilization rate of the metadata management platform is lower than the preset utilization rate;

[0114] Sort the metadata according to the individual load utilization rate of the metadata, and mark the metadata with an individual load utilization rate not higher than the load utilization rate of the metadata management platform as metadata with insufficient load utilization;

[0115] Determine the migratable load allocation amount of the metadata with insufficient load utilization according to the individual load utilization rate of the metadata, the load utilization rate of the metadata management platform, and the load allocation amount of the metadata;

[0116] Obtain the migratable load allocation metadata of the metadata with insufficient load utilization according to the data correlation between the metadata, and re-allocate the load amount of the metadata management platform in combination with the migratable load allocation amount of the metadata with insufficient load utilization.

[0117] In this embodiment, the distributed storage management result refers to the result of dividing the data storage space area according to the classification quantity of the metadata classification processing result and allocating the data storage space amount in combination with the classification management label quantity of the metadata and the data volume of the metadata, and performing distributed storage management on the metadata.

[0118] In this embodiment, the load amount of the metadata management platform refers to the total resource amount used by the metadata management platform to manage the metadata.

[0119] In this embodiment, the load allocation amount of the metadata refers to the individual resource allocation amount used by the metadata management platform to manage a single metadata.

[0120] In this embodiment, the load amount of the metadata management platform is allocated according to the distributed storage management result. For example, for the distributed storage management result c1 of metadata a1, the load amount b1 of the metadata management platform is allocated, and b1 is the load allocation amount of metadata a1.

[0121] In this embodiment, the more data storage space amount allocated by the metadata, the more load amount allocated by the metadata management platform, that is, the more load allocation amount of the metadata.

[0122] In this embodiment, the load allocation amounts of different metadata are different.

[0123] In this embodiment, the standard metadata refers to the metadata used to determine the load utilization rate of the metadata management platform, which is determined according to the metadata randomly calibrated to store data.

[0124] In this embodiment, the load utilization rate of the metadata management platform refers to the utilization rate of the metadata for the load allocated to the metadata management platform. The higher the load utilization rate of the metadata management platform, the better the metadata management is determined to be.

[0125] In this embodiment, the preset utilization rate refers to the set value set by the metadata management platform for measuring the load utilization rate of the metadata management platform.

[0126] In this embodiment, the load usage amount of the metadata refers to the actual load allocation amount used for managing the metadata.

[0127] In this embodiment, the individual load utilization rate of the metadata refers to the load utilization rate of a single metadata. For example, the load utilization rate p1 of metadata a1.

[0128] In this embodiment, the metadata with insufficient load utilization refers to the metadata whose individual load utilization rate is not higher than the load utilization rate of the metadata management platform. For example, the load utilization rate p2 of the metadata management platform, and p1 < p2, then metadata a1 is the metadata with insufficient load utilization.

[0129] In this embodiment, the migratable load allocation amount refers to the load allocation amount in the metadata with insufficient load utilization that can be used for other metadata to perform load management.

[0130] In this embodiment, the migratable load allocation amount of the metadata with insufficient load utilization is determined according to the individual load utilization rate of the metadata, the load utilization rate of the metadata management platform, and the load allocation amount of the metadata. For example, the migratable load allocation amount of metadata a1 is (p2 - p1) * the load allocation amount b1 of metadata a1.

[0131] In this embodiment, the migratable load allocation metadata of the metadata with insufficient load utilization refers to the metadata that has data correlation with the metadata with insufficient load utilization.

[0132] In this embodiment, the migratable load allocation metadata of the metadata with insufficient load utilization is obtained according to the data correlation between the metadata, and the load amount of the metadata management platform is redistributed in combination with the migratable load allocation amount of the metadata with insufficient load utilization. For example, if metadata a2 has data correlation with metadata a1, then metadata a1 can migrate the migratable load allocation amount of metadata a1 to metadata a2 to improve the load utilization rate of metadata a1.

[0133] The working principle and beneficial effects of the above technical solution are: By performing metadata management according to the data correlation between the metadata and the load amount of the metadata management platform, intelligent database metadata management is realized.

[0134] Embodiment 7:

[0135] An embodiment of the present invention provides an intelligent database metadata management system, and a metadata storage module, including:

[0136] A metadata hierarchical storage unit, configured to determine the storage level of metadata according to the load distribution amount of the metadata, and perform hierarchical storage management on the metadata. Among them, the storage levels of the metadata include: a hot metadata storage level, a general metadata storage level, and a cold metadata storage level;

[0137] A metadata clearing unit, configured to set a metadata storage management period, regularly update the storage level of the metadata, and perform metadata clearing management on the metadata at the cold metadata storage level according to the storage time.

[0138] In this embodiment, the storage level of the metadata is determined according to the load distribution amount of the metadata, and hierarchical storage management is performed on the metadata. For example, load distribution amounts a1 and a2 are set, where a1 > a2. When the load distribution amount of the metadata is not less than a1, it is determined that the storage level of the metadata is the hot metadata storage level; when the load distribution amount of the metadata is in (a2, a1), it is determined that the storage level of the metadata is the general metadata storage level; when the load distribution amount of the metadata is less than a2, it is determined that the storage level of the metadata is the cold metadata storage level.

[0139] In this embodiment, setting the metadata storage management period means setting the management period of the metadata storage management. For example, the metadata storage management period t1 is set, that is, the metadata is managed every t1 period.

[0140] In this embodiment, metadata clearing management is performed on the metadata at the cold metadata storage level according to the storage time, that is, the metadata at the cold metadata storage level with a long storage time is preferentially cleared.

[0141] The working principle and beneficial effects of the above technical solution are: By performing hierarchical storage management and clearing management on the metadata, the management efficiency of the metadata is ensured.

[0142] Embodiment 8:

[0143] An embodiment of the present invention provides an intelligent database metadata management system, further including:

[0144] A metadata visualization module, configured to visually display the metadata management result.

[0145] In this embodiment, the metadata management result is visually displayed. For example, the load management result of the metadata, the load utilization rate of the metadata management platform, and the storage level of the metadata are visually displayed.

[0146] The working principle and beneficial effects of the above technical solution are as follows: By visualizing the metadata management results, it is beneficial to ensure the user experience.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent database metadata management system, characterized in that: include: The metadata association module is used to obtain the metadata of the database and perform data analysis to determine the data correlation between the metadata; The metadata management module is used to determine the data transmission source of metadata and classify metadata, build a metadata management platform, determine the distributed architecture of metadata, and manage metadata based on the data correlation between metadata and the load of the metadata management platform; A metadata storage module is used to store and manage metadata according to metadata management results; The metadata management module includes: The load management unit is used to distribute the load of the metadata management platform according to the distributed storage management result and obtain the load distribution amount of the metadata; A metadata of the stored data is randomly calibrated as the standard metadata, and the load utilization rate of the metadata management platform is calculated according to the data correlation between other metadata and the standard metadata and the load distribution of the standard metadata; ;in, Indicates the load utilization of the metadata management platform; Indicates the load distribution amount of standard metadata; Indicates The number of occurrences of other metadata in the corresponding stored data; Indicates The number of other metadata corresponding to the stored data; Indicates The number of other metadata that appear in the metadata management platform; Indicates Data dependencies of other metadata and standard metadata; Indicates The amount of other metadata in the metadata management platform; Indicates the amount of standard metadata in the metadata management platform; Indicates the load of the metadata management platform; Indicates the number of other metadata; When the load utilization rate of the metadata management platform is not lower than the preset utilization rate, it is determined that the metadata load management is completed; A load migration management unit, used to determine a separate load utilization rate of the metadata according to a load distribution amount of the metadata and a load usage amount of the metadata when a load utilization rate of the metadata management platform is lower than a preset utilization rate; Sort the metadata according to their individual load utilization rates, and mark the metadata whose individual load utilization rate is not higher than the load utilization rate of the metadata management platform as metadata with insufficient load utilization; Determine the migratable load distribution amount of the underutilized metadata according to the individual load utilization of the metadata, the load utilization of the metadata management platform, and the load distribution amount of the metadata; According to the data correlation between metadata, the migratable load distribution metadata of the under-utilized metadata is obtained, and the load of the metadata management platform is reallocated based on the migratable load distribution amount of the under-utilized metadata.

2. An intelligent database metadata management system according to claim 1, characterized in that: Metadata association modules, including: A metadata acquisition unit, used to automatically acquire the stored data of the database and acquire the metadata of the stored data according to a preset data extraction strategy; The metadata classification unit is used to perform a metadata classification process on the metadata of the stored data according to the data type of the stored data, and classify the metadata into text metadata, image metadata and other metadata; The metadata of the stored data is subjected to secondary metadata classification processing according to the metadata type, and is divided into inherent metadata, management metadata and description metadata.

3. An intelligent database metadata management system according to claim 2, characterized in that: Metadata association modules, including: A metadata association unit, used for determining whether there is a metadata association between the stored data when there is a data link between the stored data; An association refinement unit, used to obtain metadata corresponding to the storage data with metadata association, recorded as association metadata; Performing association refinement analysis on the associated metadata according to the primary metadata classification processing result and the secondary metadata classification processing result, and when there is a data link in the associated metadata in a metadata classification processing result, determining that the metadata type corresponding to the metadata classification processing result is the association refinement type of the associated metadata; When there is no data link for the associated metadata in a metadata classification processing result, determining that the metadata type corresponding to the metadata classification processing result is an association-irrelevant type of the associated metadata; The data correlation unit is used to extract all stored data corresponding to the associated refinement type of the associated metadata to perform data correlation analysis and determine the data correlation between the metadata.

4. The intelligent database metadata management system according to claim 2, characterized in that: Metadata management module, including: A transmission source metadata classification unit is used to obtain the data transmission relationship between metadata and determine the data transmission source of the metadata; Perform tertiary metadata classification processing according to the data transmission source of metadata; The classification management unit is used to add classification management tags to the metadata for classification management of the primary metadata classification processing results, the secondary metadata classification processing results and the tertiary metadata classification processing results, wherein the classification management tags include: primary metadata classification management tags, secondary metadata classification management tags and tertiary metadata classification management tags.

5. An intelligent database metadata management system according to claim 4, characterized in that: Metadata management module, including: The platform construction unit is used to obtain metadata management requirements, build a metadata management platform, and determine the distributed architecture of metadata; The distributed storage management unit is used to obtain the data storage space of metadata according to the distributed architecture of metadata, divide the data storage space area according to the number of categories of the metadata classification processing results, and allocate the data storage space according to the number of metadata classification management tags and the data volume of metadata, so as to perform distributed storage management of metadata.

6. The intelligent database metadata management system according to claim 1, characterized in that: Metadata storage module, including: A metadata hierarchical storage unit, used to determine the storage level of metadata according to the load distribution amount of metadata, and perform hierarchical storage management on metadata, wherein the metadata storage level includes: hot spot metadata storage level, general metadata storage level and cold spot metadata storage level; The metadata clearing unit is used to set the metadata storage management cycle, regularly update the metadata storage level and perform metadata clearing management on the metadata of the cold spot metadata storage level according to the storage time.

7. The intelligent database metadata management system according to claim 1, characterized in that: Also includes: The metadata visualization module is used to visualize the metadata management results.

Images