A method and system for generation of metadata for a heterogeneous system

Abstract

The application relates to the technical field of data processing, in particular to a metadata generation method and system for a heterogeneous system, which comprises the following steps: obtaining a metadata sequence and a corresponding metadata access sequence of the heterogeneous system, calculating a metadata heat sequence corresponding to the metadata sequence based on the metadata sequence and the metadata access sequence, predicting a metadata heat sequence at a next preset time based on the metadata heat sequence, and classifying the metadata heat sequence at the next preset time based on a preset heat threshold to obtain a first metadata heat sequence and a second metadata heat sequence. Through the prediction and classification of the metadata heat, the system can optimize the indexing of the metadata, thereby accelerating the retrieval speed, and the metadata can be classified and clustered according to the heat sequence, which is helpful for the centralized storage of frequently accessed data, reduces the waste of storage resources, and optimizes the storage layout.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a metadata generation method and system for a heterogeneous system. BACKGROUND

[0002] A heterogeneous system refers to a system composed of different types of hardware, software or networks, which are usually used to process diversified data sources and computing tasks, improve the performance and flexibility of the system, and may have different architectures, platforms, data formats and communication protocols.

[0003] The traditional method does not have an index mechanism optimized for heat, resulting in a long data retrieval time; and the data is not classified and clustered in the traditional method, resulting in non-optimal allocation of storage resources; and the traditional method cannot dynamically adjust the index and storage layout according to the data access mode and heat changes; and the traditional method has a fixed management and organization of data structure, which cannot effectively cope with frequently changing data environments. SUMMARY

[0004] The present application aims to solve the problems in the background art and provides a metadata generation method and system for a heterogeneous system.

[0005] The technical scheme of the present application: a metadata generation method for a heterogeneous system, comprising:

[0006] Obtaining a metadata sequence and a corresponding metadata access sequence of a heterogeneous system, and calculating a metadata heat sequence corresponding to the metadata sequence based on the metadata sequence and the metadata access sequence;

[0007] Based on the metadata heat sequence, predicting a metadata heat sequence at a next preset time, and based on a preset heat threshold, classifying the metadata heat sequence at the next preset time to obtain a first metadata heat sequence and a second metadata heat sequence;

[0008] Based on the first metadata heat sequence and the second metadata heat sequence, obtaining a first metadata sequence and a second metadata sequence, and clustering the first metadata sequence and the second metadata sequence to obtain a plurality of first clustering clusters corresponding to the first metadata sequence and a plurality of second clustering clusters corresponding to the second metadata sequence;

[0009] Feature extraction is performed on the plurality of first clustering clusters and the plurality of second clustering clusters to obtain a plurality of first data features and a plurality of second data features, and a similarity model is used to calculate a first similarity between the plurality of first data features and a second similarity between the plurality of second data features;

[0010] The first clustering clusters are sorted according to the size of the first similarity to obtain a first clustering cluster sequence, and the second clustering clusters are sorted according to the size of the second similarity to obtain a second clustering cluster sequence, the first clustering cluster sequence and the second clustering cluster sequence are combined to obtain a clustering cluster sequence.

[0011] A B+ tree is used based on the order of the clustering cluster sequence to establish an index of each clustering cluster with a file mark as a key.

[0012] Preferably, the hotness calculation formula of the metadata is as follows:

[0013]

[0014] Wherein, H j represents the hotness when the metadata is accessed for the jth time, m represents the access time when the metadata is accessed for the mth time when the jth access request for the metadata is received, t j represents the access time when the metadata is accessed for the jth time, t j-m represents the access time when the metadata is accessed for the j-mth time.

[0015] Preferably, the metadata hotness sequence is predicted based on the metadata hotness sequence at the next preset time, comprising:

[0016] The metadata hotness sequence is accumulated to obtain an accumulated metadata hotness sequence, and the accumulated metadata hotness sequence is expressed as follows:

[0017] h (1 )=[h (1) (k1),h (1) (k2),...,h (1 )(k i ),...,h( 1) (k n )];

[0018] Wherein, h (1) represents the accumulated metadata hotness sequence, h 1 )(k i ) represents the i th accumulated metadata hotness in the accumulated metadata hotness sequence, and h 0 )(k j ) represents the j th metadata hotness in the metadata hotness sequence, Δk j represents the interval of the metadata hotness sequence, and Δk j =k j -k j-1 ;

[0019] constructing a gray prediction model for the accumulated metadata hotness sequence, the gray prediction model being as follows:

[0020]

[0021] wherein a represents a development coefficient, and b represents a gray action amount.

[0022] Preferably, predicting the metadata hotness sequence at the next preset time based on the metadata hotness sequence comprises:

[0023] constructing a whitening response formula based on the gray prediction model, the whitening response formula being as follows:

[0024]

[0025] wherein, represents the whitening response formula of the gray prediction model when t=k1, and and represents the background value of the accumulated metadata hotness sequence in the interval [k i ,k i+1 ], and and

[0026] restoring the metadata hotness sequence based on the gray prediction model of the gray prediction model to obtain the metadata hotness sequence at the next preset time, the metadata hotness sequence at the next preset time being expressed as follows:

[0027]

[0028] wherein, represents the metadata hotness sequence at the next preset time.

[0029] Preferably, clustering the first metadata sequence and the second metadata sequence to obtain a plurality of first clustering clusters corresponding to the first metadata sequence and a plurality of second clustering clusters corresponding to the second metadata sequence comprises:

[0030] predefining a state value function, determining the clustering stop point of the first metadata sequence and the second metadata sequence through the state value function, the value function being expressed as follows:

[0031]

[0032] wherein δ represents the state value function, θ represents the Euclidean distance, and χ p represents a data object including a data attribute p, A data center representing a correlation rule u, u represents a certain correlation rule, P represents a data attribute set, and p represents a data attribute;

[0033] Calculate the membership degree of each element of the first metadata sequence and the second metadata sequence based on a fuzzy mean algorithm and a relation matrix;

[0034] Construct a target function with the minimum membership degree as the target, set a constraint condition for the state value function, solve the target function and the constraint condition by a Lagrange multiplier method to obtain a plurality of first clustering clusters corresponding to the first metadata sequence and a plurality of second clustering clusters corresponding to the second metadata sequence, and the constraint condition is as follows:

[0035]

[0036] Wherein, s.t represents a constraint condition, h represents data to be clustered, u ph represents the correlation degree between the data to be clustered h and the data attribute p, and ε represents the total number of data to be clustered.

[0037] Preferably, feature extraction is performed on the plurality of first clustering clusters and the plurality of second clustering clusters to obtain a plurality of first data features and a plurality of second data features, and a similarity model is used to calculate a first similarity between the plurality of first data features and a second similarity between the plurality of second data features, including:

[0038] For each of the first clustering cluster and the second clustering cluster, a principal component analysis method is used to extract a first feature vector of the first clustering cluster and a second feature vector of the second clustering cluster;

[0039] A similarity model is used to calculate a first similarity between each pair of the first feature vectors and a second similarity between each pair of the second feature vectors to obtain a first similarity sequence and a second similarity sequence.

[0040] Preferably, the plurality of first clustering clusters are sorted according to the size of the first similarity to obtain a first clustering cluster sequence, and the plurality of second clustering clusters are sorted according to the size of the second similarity to obtain a second clustering cluster sequence, and the first clustering cluster sequence and the second clustering cluster sequence are combined to obtain a clustering cluster sequence, including:

[0041] The first similarity sequence and the second similarity sequence are sorted to obtain a first similarity order sequence and a second similarity order sequence;

[0042] sequencing a plurality of first clustering clusters based on the first similarity order sequence to obtain a first clustering cluster sequence, and sequencing a plurality of second clustering clusters based on the second similarity order sequence to obtain a second clustering cluster sequence;

[0043] combining the second clustering cluster sequence into the first clustering cluster sequence, and combining a first element in the second clustering cluster sequence into a last element in the first clustering cluster sequence to obtain a clustering cluster sequence.

[0044] Preferably, the similarity model adopts cosine similarity.

[0045] Preferably, a B+ tree is adopted to establish an index of each clustering cluster based on the order of the clustering cluster sequence, with a file mark as a key, including:

[0046] determining the order of the B+ tree, and initializing a root node of the B+ tree as empty;

[0047] inserting a file mark of each clustering cluster into the B+ tree as a key, and updating an internal node in the process of insertion, and splitting if the internal node exceeds a maximum capacity, and ascending a middle key to a parent node, and creating a new internal node;

[0048] storing the file mark of the clustering cluster in a leaf node of the B+ tree, and keeping the leaf node sorted by the key, and splitting the leaf node if the leaf node exceeds the maximum capacity, and ascending the middle key to the parent node, and creating a new leaf node;

[0049] when the clustering cluster sequence is traversed, a new B+ tree is obtained, and an index of each clustering cluster is established based on the new B+ tree.

[0050] The technical scheme of the application: a metadata generation system for a heterogeneous system, which is suitable for the metadata generation method for the heterogeneous system, and includes:

[0051] a data acquisition unit, which is configured to acquire a metadata sequence and a corresponding metadata access sequence of a heterogeneous system, and calculate a metadata hotness sequence corresponding to the metadata sequence based on the metadata sequence and the metadata access sequence;

[0052] a hotness prediction unit, which is configured to predict a metadata hotness sequence at a next preset time based on the metadata hotness sequence, and classify the metadata hotness sequence at the next preset time based on a preset hotness threshold to obtain a first metadata hotness sequence and a second metadata hotness sequence;

[0053] a data clustering unit, configured to obtain a first metadata sequence and a second metadata sequence based on the first metadata hotness sequence and the second metadata hotness sequence, and cluster the first metadata sequence and the second metadata sequence to obtain a plurality of first clustering clusters corresponding to the first metadata sequence and a plurality of second clustering clusters corresponding to the second metadata sequence;

[0054] a similarity calculation unit, configured to extract features from the plurality of first clustering clusters and the plurality of second clustering clusters to obtain a plurality of first data features and a plurality of second data features, and calculate first similarities between the plurality of first data features and second similarities between the plurality of second data features by using a similarity model;

[0055] a clustering combination unit, configured to sort the plurality of first clustering clusters according to the sizes of the first similarities to obtain a first clustering cluster sequence, sort the plurality of second clustering clusters according to the sizes of the second similarities to obtain a second clustering cluster sequence, and combine the first clustering cluster sequence and the second clustering cluster sequence to obtain a clustering cluster sequence;

[0056] an index establishment unit, configured to establish indexes of the respective clustering clusters by using a B+ tree with file marks as keys based on the order of the clustering cluster sequence.

[0057] Compared with the prior art, the above technical solution of the present application has the following beneficial technical effects:

[0058] By predicting and classifying metadata hotness, the system can optimize the indexing of metadata, thereby speeding up the retrieval speed, and classifying and clustering metadata according to the hotness sequence can help to store frequently accessed data in a centralized manner, reduce the waste of storage resources, and optimize the storage layout. Moreover, through classification and sorting based on the hotness sequence, the system can respond more quickly to the user's demand for high-frequency access data, and using the B+ tree structure to establish indexes can effectively support dynamic updating and efficient data access. Furthermore, by clustering and extracting features from metadata, the system can reveal the internal structure and relationship of data, which helps to more effectively organize and manage metadata, improve the availability and consistency of data, and through the order of the clustering cluster and the index structure, the system can dynamically adjust according to the changes of data, thereby adapting to different access modes and data changes. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 a step flowchart of the overall method in an embodiment of the present application;

[0060] Figure 2 a system architecture diagram of the overall system in an embodiment of the present application.

[0061] 1, data acquisition unit; 2, hotness prediction unit; 3, data clustering unit; 4, similarity calculation unit; 5, clustering combination unit; 6, index establishment unit. DETAILED DESCRIPTION

[0062] Embodiment one, as shown, the present application proposes a kind of generation method of metadata for heterogeneous system, comprising: Figure 1

[0063] S1, obtain the metadata sequence and corresponding metadata access sequence of heterogeneous system, based on metadata sequence and metadata access sequence, the metadata hotness sequence corresponding to metadata sequence is calculated;

[0064] S2, based on metadata hotness sequence, the metadata hotness sequence of next preset time is predicted, and based on the preset hotness threshold, the metadata hotness sequence of next preset time is classified, to obtain first metadata hotness sequence and second metadata hotness sequence;

[0065] S3, based on first metadata hotness sequence and second metadata hotness sequence, obtain first metadata sequence and second metadata sequence, the first metadata sequence and second metadata sequence are clustered, to obtain the multiple first clustering clusters corresponding to first metadata sequence and the multiple second clustering clusters corresponding to second metadata sequence;

[0066] S4, the feature extraction of multiple first clustering clusters and multiple second clustering clusters is carried out, to obtain multiple first data features and multiple second data features, the first similarity between multiple first data features and the second similarity between multiple second data features are calculated by similarity model;

[0067] S5, multiple first clustering clusters are sorted according to the size of first similarity, to obtain first clustering cluster sequence, and multiple second clustering clusters are sorted according to the size of second similarity, to obtain second clustering cluster sequence, first clustering cluster sequence and second clustering cluster sequence are combined, to obtain clustering cluster sequence;

[0068] S6, B+ tree is used based on the order of clustering cluster sequence with file mark as key to establish the index of each clustering cluster respectively.

[0069] ​In the present application, the metadata sequence refers to a sequence of information describing the structure, attributes or relationships of data; the metadata access sequence refers to a sequence recording the access records of metadata; the metadata heat sequence refers to a sequence of metadata importance calculated based on the access frequency; the heat threshold refers to a preset limit value for classifying the metadata heat sequence; the clustering refers to a process of grouping data into several similar data sets; the B+ tree refers to a balanced tree data structure for efficient data retrieval and sorting; and the index refers to a structure for organizing data to speed up queries.

[0070] In the second embodiment, the present application proposes a metadata generation method for a heterogeneous system, which, compared with the first embodiment, further comprises: a heat calculation formula of the metadata is as follows:

[0071]

[0072] wherein, H j represents the heat when the metadata is accessed for the jth time, m represents the access time when the metadata is accessed for the mth time before the jth access request for the metadata is received, t j represents the access time when the metadata is accessed for the jth time, t j-m represents the access time when the metadata is accessed for the j-mth time.

[0073] In an optional embodiment, the metadata heat sequence at a next preset time is predicted based on the metadata heat sequence, comprising:

[0074] A1, the metadata heat sequence is accumulated to obtain an accumulated metadata heat sequence, and the accumulated metadata heat sequence is expressed as follows:

[0075] h (1) =[h (1) (k1),h (1) (k2),...,h (1) (k i ),...,h (1) (k n )];

[0076] wherein, h (1) represents the accumulated metadata heat sequence, h (1) (k i ) represents the ith accumulated metadata heat in the accumulated metadata heat sequence, and h (0) (k j ) represents the jth metadata heat in the metadata heat sequence, Δk j represents the interval of the metadata heat sequence, and Δk j =k j -k j-1 ;

[0077] A2, a gray prediction model is constructed for the accumulated metadata heat sequence, and the gray prediction model is as follows:

[0078]

[0079] wherein a represents a development coefficient, and b represents a gray action amount.

[0080] It should be noted that the gray prediction model refers to a prediction method based on the gray system theory, which is used to process incomplete information and predict the change trend of data through modeling and analysis; the development coefficient refers to a development coefficient in the gray prediction model, which reflects the trend of data growth and helps to determine the predicted value of future data; and the gray action amount refers to an amount for measuring the cumulative effect of data change, which is used to analyze and calculate the trend and fluctuation of data in the gray model.

[0081] In an optional embodiment, the metadata heat sequence at the next preset time is predicted based on the metadata heat sequence, and the method further comprises:

[0082] A3, a whitening response formula is constructed based on the gray prediction model, and the whitening response formula is as follows:

[0083]

[0084] wherein, represents the whitening response formula of the gray prediction model when t=k1, and and represents the background value of the accumulated metadata heat sequence in the interval [k i , k i+1 ], and and

[0085] A4, the metadata heat sequence is restored based on the gray prediction model of the gray prediction model, to obtain the metadata heat sequence at the next preset time, and the expression of the metadata heat sequence at the next preset time is as follows:

[0086]

[0087] wherein, represents the metadata heat sequence at the next preset time.

[0088] It should be noted that whitening aims to convert data into white noise, in this case, the covariance matrix of the data becomes an identity matrix, the purpose of whitening is to remove the correlation in the data, so that the data has more uniform statistical properties; response usually refers to the reaction of data to certain inputs or external factors, in the whitening process, the concept of response may involve how to make the data processing or modeling process more sensitive or more adaptive to different input changes.

[0089] In an optional embodiment, clustering the first metadata sequence and the second metadata sequence to obtain a plurality of first clustering clusters corresponding to the first metadata sequence and a plurality of second clustering clusters corresponding to the second metadata sequence comprises:

[0090] B1, defining a state value function in advance, determining a clustering stop point of the first metadata sequence and the second metadata sequence through the state value function, and the value function expression is as follows:

[0091]

[0092] Wherein, δ represents the state value function, θ represents, η represents the Euclidean distance, χ p represents a data object including a data attribute p, represents a data center related to an association rule u, u represents a certain association rule, P represents a data attribute set, and p represents a data attribute;

[0093] B2, calculating the membership degree of each element of the first metadata sequence and the second metadata sequence based on the fuzzy mean algorithm and the relationship matrix;

[0094] B3, constructing a target function with the minimum membership degree as the target, setting a constraint condition for the state value function, and solving the target function and the constraint condition through the Lagrange multiplier method to obtain a plurality of first clustering clusters corresponding to the first metadata sequence and a plurality of second clustering clusters corresponding to the second metadata sequence, and the constraint condition is as follows:

[0095]

[0096] Wherein, s.t represents the constraint condition, h represents the data to be clustered, u ph represents the association degree between the data to be clustered h and the data attribute p, and ε represents the total number of data to be clustered.

[0097] In an optional embodiment, feature extraction is performed on the plurality of first clustering clusters and the plurality of second clustering clusters to obtain a plurality of first data features and a plurality of second data features, and a first similarity between the plurality of first data features and a second similarity between the plurality of second data features are calculated through a similarity model, comprising:

[0098] C1, for each first clustering cluster and each first clustering cluster, extracting a first feature vector of the first clustering cluster and a second feature vector of the second clustering cluster by a principal component analysis method;

[0099] C2, calculating a first similarity between each pair of first feature vectors and a second similarity between each pair of second feature vectors by a similarity model, to obtain a first similarity sequence and a second similarity sequence.

[0100] In an optional embodiment, the plurality of first clustering clusters are sorted according to the size of the first similarity to obtain a first clustering cluster sequence, and the plurality of second clustering clusters are sorted according to the size of the second similarity to obtain a second clustering cluster sequence, and the first clustering cluster sequence and the second clustering cluster sequence are combined to obtain a clustering cluster sequence, comprising:

[0101] D1, sorting the first similarity sequence and the second similarity sequence to obtain a first similarity order sequence and a second similarity order sequence;

[0102] D2, sorting the plurality of first clustering clusters based on the first similarity order sequence to obtain a first clustering cluster sequence, and sorting the plurality of second clustering clusters based on the second similarity order sequence to obtain a second clustering cluster sequence;

[0103] D3, combining the second clustering cluster sequence into the first clustering cluster sequence, and combining the first element in the second clustering cluster sequence to the last element in the first clustering cluster sequence to obtain a clustering cluster sequence.

[0104] In an optional embodiment, the similarity model adopts cosine similarity.

[0105] It should be noted that the cosine similarity is a measure of the angle between two vectors in a vector space, which is mainly used to evaluate the directional similarity of two vectors, rather than their size or length, and the cosine similarity is commonly used in text analysis and recommendation systems, especially when dealing with high-dimensional sparse data.

[0106] In an optional embodiment, a B+ tree is adopted based on the order of the clustering cluster sequence to establish an index of each clustering cluster with a file mark as a key, comprising:

[0107] E1, determining the order of the B+ tree, and initializing the root node of the B+ tree as empty;

[0108] E2, inserting the file mark of each clustering cluster as a key into the B+ tree, and in the insertion process, updating the internal node, if the internal node exceeds the maximum capacity, splitting, and raising the intermediate key to the parent node, and creating a new internal node;

[0109] E3, storing file marks of the clustering cluster in the leaf node of the B+ tree, and keeping the leaf node sorted by key, if the leaf node exceeds the maximum capacity, splitting the leaf node, and moving the middle key to the parent node, and creating a new leaf node;

[0110] E4, after the clustering cluster sequence traversal is completed, a new B+ tree is obtained, and indexes of various clustering clusters are established based on the new B+ tree.

[0111] It should be noted that the order is an important parameter in the B+ tree, indicating how many child nodes (for internal nodes) or how many keys (for leaf nodes) a node can contain at most; the root node is the top node of the B+ tree, the starting node of the tree, and when initialized, the root node is empty; insertion means adding new keys to the B+ tree; splitting means splitting a node into two nodes, when an internal node is split, part of the keys are moved to the newly created internal node, and the middle key is moved to the parent node, when a leaf node is split, the keys are also divided into new leaf nodes, and the keys are sorted; in the splitting process of the B+ tree, the middle key is moved to the parent node, the parent node can continue to maintain the balance of the tree, and provide a suitable split point for the child node; the index is a data structure used to improve the efficiency of data retrieval. In the B+ tree, the index usually refers to the structure based on the key (such as file mark), which helps to quickly find and access data.

[0112] Embodiment three, as shown in the Figure 2 The application provides a metadata generation system for a heterogeneous system, which is suitable for the metadata generation method for the heterogeneous system and comprises:

[0113] The data acquisition unit 1 is used for acquiring the metadata sequence and the corresponding metadata access sequence of the heterogeneous system, calculating the metadata heat sequence corresponding to the metadata sequence based on the metadata sequence and the metadata access sequence;

[0114] The heat prediction unit 2 is used for predicting the metadata heat sequence at the next preset time based on the metadata heat sequence, and classifying the metadata heat sequence at the next preset time based on the preset heat threshold, so as to obtain the first metadata heat sequence and the second metadata heat sequence;

[0115] The data clustering unit 3 is used for acquiring the first metadata sequence and the second metadata sequence based on the first metadata heat sequence and the second metadata heat sequence, and clustering the first metadata sequence and the second metadata sequence, so as to obtain a plurality of first clustering clusters corresponding to the first metadata sequence and a plurality of second clustering clusters corresponding to the second metadata sequence;

[0116] The similarity calculation unit 4 is configured to perform feature extraction on the first and second clustering groups to obtain first and second data features, and calculate first and second similarities between the first and second data features by using a similarity model.

[0117] The clustering combination unit 5 is configured to sort the first clustering groups according to the first similarity to obtain a first clustering group sequence, sort the second clustering groups according to the second similarity to obtain a second clustering group sequence, and combine the first and second clustering group sequences to obtain a clustering group sequence.

[0118] The index establishment unit 6 is configured to establish an index of each clustering group by using a B+ tree with a file mark as a key based on the order of the clustering group sequence.

[0119] The above detailed description of the embodiments of the present application is made in combination with the accompanying drawings, but the present application is not limited thereto, and various changes can be made within the knowledge of those skilled in the art without departing from the spirit of the present application.

Claims

1. A method for generating metadata for a heterogeneous system, characterized in that, The method comprises the following steps: obtaining a metadata sequence and a corresponding metadata access sequence of a heterogeneous system, calculating a metadata heat sequence corresponding to the metadata sequence based on the metadata sequence and the metadata access sequence; predicting a metadata heat sequence at a next preset time based on the metadata heat sequence, and classifying the metadata heat sequence at the next preset time based on a preset heat threshold to obtain a first metadata heat sequence and a second metadata heat sequence; obtaining a first metadata sequence and a second metadata sequence based on the first metadata heat sequence and the second metadata heat sequence, and clustering the first metadata sequence and the second metadata sequence to obtain a plurality of first clustering clusters corresponding to the first metadata sequence and a plurality of second clustering clusters corresponding to the second metadata sequence; extracting features of the plurality of first clustering clusters and the plurality of second clustering clusters to obtain a plurality of first data features and a plurality of second data features, and calculating a first similarity between the plurality of first data features and a second similarity between the plurality of second data features through a similarity model; sorting the plurality of first clustering clusters according to the size of the first similarity to obtain a first clustering cluster sequence, and sorting the plurality of second clustering clusters according to the size of the second similarity to obtain a second clustering cluster sequence, and combining the first clustering cluster sequence and the second clustering cluster sequence to obtain a clustering cluster sequence; establishing an index of each clustering cluster based on the order of the clustering cluster sequence using a B+ tree with a file mark as a key.

2. The method for metadata generation for a heterogeneous system according to claim 1, wherein, The heat calculation formula of the metadata is as follows: ； in, Indicates the popularity of metadata when it is accessed for the jth time, Indicates the access time of the most recent m metadata accesses when the j-th access request to the metadata is received. represents the access time when the metadata is accessed for the jth time, Indicates the access time when the jmth metadata access was made.

3. The method for metadata generation for a heterogeneous system according to claim 2, wherein, predicting a metadata heat sequence at a next preset time based on the metadata heat sequence, comprising: accumulating the metadata heat sequence to obtain an accumulated metadata heat sequence, and the expression of the accumulated metadata heat sequence is as follows: ； wherein, represents an accumulated metadata hotness sequence, represents the i-th accumulated metadata hotness in the accumulated metadata hotness sequence, and , represents the j-th metadata hotness in the metadata hotness sequence, represents a spacing of the metadata hotness sequence, and ; constructing a gray prediction model for the accumulated metadata heat sequence, and the gray prediction model is as follows: ； wherein, represents a development coefficient, represents a gray action amount.

4. The method for metadata generation for a heterogeneous system according to claim 3, wherein, predicting a metadata heat sequence at a next preset time based on the metadata heat sequence, further comprising: constructing a whitening response formula based on the gray prediction model, and the whitening response formula is as follows: ； wherein, represents a whitening response formula of the gray scale prediction model when , and , and , represents a background value of the cumulative metadata hotness sequence in the interval , and , and ; restoring the metadata heat sequence based on the whitening response formula of the gray prediction model to obtain a metadata heat sequence at a next preset time, and the expression of the metadata heat sequence at the next preset time is as follows: ； wherein, represents a metadata hotness sequence of the next preset time.

5. The method for metadata generation for a heterogeneous system of claim 1, wherein, clustering the first metadata sequence and the second metadata sequence to obtain a plurality of first clustering clusters corresponding to the first metadata sequence and a plurality of second clustering clusters corresponding to the second metadata sequence, comprising: predefining a state value function to determine the clustering stop point of the first metadata sequence and the second metadata sequence through the state value function, and the expression of the state value function is as follows: ； wherein denotes a state value function, denotes the total number of association rules, denotes the Euclidean distance, denotes a data object comprising data attributes , denotes a data center on association rules , denotes a certain association rule, denotes a data attribute set, denotes a data attribute; calculating the belonging degree of each element of the first metadata sequence and the second metadata sequence based on a fuzzy mean algorithm and a relationship matrix; A target function is constructed with the minimum of the homogeneity as a target, a constraint condition is set for the state value function, the target function and the constraint condition are solved by a Lagrange multiplier method, to obtain a plurality of first clustering clusters corresponding to the first metadata sequence and a plurality of second clustering clusters corresponding to the second metadata sequence, and the constraint condition is as follows: ； wherein, represents a constraint condition, represents data to be clustered, represents a degree of association between data to be clustered h and data attribute p, represents the total number of data to be clustered.

6. The method for metadata generation for a heterogeneous system of claim 1, wherein, Feature extraction is performed on the plurality of first clustering clusters and the plurality of second clustering clusters to obtain a plurality of first data features and a plurality of second data features, and a similarity model is used to calculate a first similarity between the plurality of first data features and a second similarity between the plurality of second data features, including: For each of the first clustering cluster and each of the first clustering cluster, a principal component analysis method is used to extract a first feature vector of the first clustering cluster and a second feature vector of the second clustering cluster; The first similarity between each pair of the first feature vectors and the second similarity between each pair of the second feature vectors are calculated by a similarity model to obtain a first similarity sequence and a second similarity sequence.

7. The method for metadata generation for a heterogeneous system according to claim 6, wherein, The plurality of first clustering clusters are sorted according to the size of the first similarity to obtain a first clustering cluster sequence, and the plurality of second clustering clusters are sorted according to the size of the second similarity to obtain a second clustering cluster sequence, and the first clustering cluster sequence and the second clustering cluster sequence are combined to obtain a clustering cluster sequence, including: The first similarity sequence and the second similarity sequence are sorted to obtain a first similarity order sequence and a second similarity order sequence; The plurality of first clustering clusters are sorted based on the first similarity order sequence to obtain a first clustering cluster sequence, and the plurality of second clustering clusters are sorted based on the second similarity order sequence to obtain a second clustering cluster sequence; The second clustering cluster sequence is combined into the first clustering cluster sequence, and the first element in the second clustering cluster sequence is combined into the last element in the first clustering cluster sequence to obtain a clustering cluster sequence.

8. The method for metadata generation for a heterogeneous system of claim 6, wherein, The similarity model uses cosine similarity.

9. The method for metadata generation for a heterogeneous system of claim 1, wherein, A B+ tree is used to establish an index of each clustering cluster based on the order of the clustering cluster sequence with a file mark as a key, including: The order of the B+ tree is determined, and the root node of the B+ tree is initialized as empty; The file mark of each clustering cluster is inserted into the B+ tree as a key, and in the insertion process, the internal node is updated, if the internal node exceeds the maximum capacity, splitting is performed, the middle key is raised to the parent node, and a new internal node is created; The file mark of the clustering cluster is stored in the leaf node of the B+ tree, and the leaf node is kept sorted by key, if the leaf node exceeds the maximum capacity, the leaf node is split, the middle key is raised to the parent node, and a new leaf node is created; When the clustering cluster sequence is traversed, a new B+ tree is obtained, and the index of each clustering cluster is established based on the new B+ tree.

10. A metadata generation system for a heterogeneous system, which is adapted to the metadata generation method for a heterogeneous system according to any one of claims 1 to 9, characterized in that, Including: The data acquisition unit (1) is configured to acquire a metadata sequence and a corresponding metadata access sequence of a heterogeneous system, and calculate a metadata hotness sequence corresponding to the metadata sequence based on the metadata sequence and the metadata access sequence; The hotness prediction unit (2) is configured to predict a metadata hotness sequence at a next preset time based on the metadata hotness sequence, and classify the metadata hotness sequence at the next preset time based on a preset hotness threshold to obtain a first metadata hotness sequence and a second metadata hotness sequence; The data clustering unit (3) is configured to acquire a first metadata sequence and a second metadata sequence based on the first metadata hotness sequence and the second metadata hotness sequence, and cluster the first metadata sequence and the second metadata sequence to obtain a plurality of first clustering clusters corresponding to the first metadata sequence and a plurality of second clustering clusters corresponding to the second metadata sequence; The similarity calculation unit (4) is configured to perform feature extraction on the plurality of first clustering clusters and the plurality of second clustering clusters to obtain a plurality of first data features and a plurality of second data features, and calculate a first similarity between the plurality of first data features and a second similarity between the plurality of second data features through a similarity model; The clustering combination unit (5) is configured to sort the plurality of first clustering clusters according to the size of the first similarity to obtain a first clustering cluster sequence, and sort the plurality of second clustering clusters according to the size of the second similarity to obtain a second clustering cluster sequence, and combine the first clustering cluster sequence and the second clustering cluster sequence to obtain a clustering cluster sequence; The index establishment unit (6) is configured to establish an index of each clustering cluster by taking a file mark as a key based on the order of the clustering cluster sequence by using a B+ tree.

Images