A metadata classification method based on machine learning algorithm

Through the metadata hierarchical classification method based on machine learning algorithm, the problem of difficulty in classifying metadata fields without unified naming rules is solved, the precise classification of metadata and data privacy protection is realized, and classification efficiency is improved.

CN114676253BActive Publication Date: 2024-11-26SHANGHAI UNIVERSITY OF ELECTRIC POWER +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210300625.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-25
Publication Date
2024-11-26
Estimated Expiration
2042-03-25

AI Technical Summary

Technical Problem

Existing text data classification methods cannot accurately classify metadata fields without unified naming rules, resulting in inefficient data classification grading and difficulty in protecting data privacy.

Method used

Using a metadata hierarchical classification method based on machine learning algorithms, we can build original metadata sets, create frequent item thesaurus, convert class text features into numerical features, and build a two-stage classification model, including binary classifiers and multiclassifiers , perform hierarchical classification prediction and sub-categorization.

Benefits of technology

It realizes the precise classification of complex text metadata, reduces workload, improves classification efficiency, and effectively protects data privacy and meets business needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114676253B_ABST
    Figure CN114676253B_ABST
Patent Text Reader

Abstract

The present invention provides a metadata hierarchical classification method based on a machine learning algorithm, comprising the following steps: first, a frequent term word library is created based on an original metadata set; then, the features of the complex and non-uniformly regular quasi-text fields in the original metadata set are innovatively converted into numerical features based on the frequent term word library, thereby solving the problem that the existing text classification method cannot accurately classify metadata fields without uniform naming rules; secondly, a two-stage classification model is constructed, and the model is trained and optimized, and the metadata to be tested is first classified by sensitivity level through the binary classifier of the model, and finally the sensitive metadata is further classified by multiple classifiers, and a more accurate hierarchical classification result is output. The method solves the problem that the classification and grading of sensitive data in the current financial field is extremely expensive due to manpower, realizes accurate classification, effectively protects data privacy, meets business needs, and improves classification work efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of metadata classification and categorization, and in particular relates to a metadata classification and categorization method based on a machine learning algorithm. Background Art

[0002] Data classification and grading management is not only a prerequisite for strengthening data exchange and sharing and improving the value of data resources, but also a necessary condition for data security protection scenarios. At the same time, data classification and grading is also a way to present data assets and a solution for data governance. Data classification is more from a business perspective, clearly knowing which business category the metadata belongs to, helping the business to clearly grasp the data dynamics and improve the ability to use and utilize data.

[0003] A lot of business data of the company is stored in a structured database, which can be classified according to the relevant fields in certain tables. Common categories include: password, customer number, ID number, address, etc. After classification, these data can be divided into importance and sensitivity according to relevant regulations to provide protection for data security. The field names of the text-like metadata used to describe the data content are generally named by business personnel. Naming itself is a very difficult thing. The rules are not unified and there is great subjectivity. For example, the naming rules include English words, English word abbreviations, numbers, and Chinese pinyin initials at the same time. However, existing text data classification methods are usually unable to accurately classify metadata fields without unified naming rules. Summary of the invention

[0004] In order to solve the above problems, a hierarchical classification method is provided to accurately classify quasi-text metadata and protect data privacy. The present invention adopts the following technical solutions:

[0005] The present invention provides a metadata hierarchical classification method based on a machine learning algorithm, which is characterized by comprising the following steps: step S1, constructing an original metadata data set, and using stratified sampling to divide the original metadata data set into a training set and a test set; step S2, creating a frequent term word library based on the training set; step S3, representing the quasi-text features of the original metadata with numerical features based on the frequent terms in the frequent term word library; step S4, constructing a two-stage classification model, using the training set represented by the numerical features to train the two-stage classification model to obtain a hierarchical classification model; step S5, inputting the metadata to be tested into the hierarchical classification model, and the hierarchical classification model performs hierarchical prediction and classification calculation on the metadata to be tested to obtain corresponding hierarchical classification results, the two-stage classification model includes a binary classifier in stage one and a multi-classifier in stage two, the binary classifier is used to perform large-category hierarchical prediction on the metadata to be tested, and the multi-classifier performs multi-classification on the hierarchical prediction results to obtain corresponding sub-category results.

[0006] In the metadata hierarchical classification method based on machine learning algorithm provided by the present invention, it can also have such a technical feature, wherein the construction process of the original metadata data set is: based on a structured database, financial metadata is collected, and the financial metadata is a text-like field, and financial metadata with different labels are classified into sensitive metadata categories and non-sensitive metadata categories, thereby constructing an original metadata data set, and using stratified sampling to divide the original metadata data set into a training set and a test set.

[0007] In the metadata hierarchical classification method based on machine learning algorithm provided by the present invention, there may also be such a technical feature, wherein the labels of financial metadata belonging to the general category of sensitive metadata include at least account, bank account number, ID number, password, name, address, telephone number and email address.

[0008] The metadata hierarchical classification method based on machine learning algorithm provided by the present invention may also have such a technical feature, wherein the frequent term word library is created in the following process: according to the frequency of occurrence of the constituent fields of the financial metadata, binary frequent terms, ternary frequent terms and single frequent terms in the class text field are extracted from the training set, and the frequent term word library corresponding to the class text field is obtained by merging these three frequent terms.

[0009] In the metadata hierarchical classification method based on machine learning algorithm provided by the present invention, there may also be such a technical feature, wherein the numerical feature is represented as follows: for each frequent item, it is determined whether each class text field in the training set contains the frequent item, thereby creating a (0-1) numerical feature corresponding to the class text field, where the value 0 indicates that the class text field does not contain the frequent item, and the value 1 indicates that the class text field contains the frequent item.

[0010] In the metadata hierarchical classification method based on machine learning algorithm provided by the present invention, it can also have such technical features, wherein, the binary classifier is constructed by using the Bagging model integration method based on a predetermined algorithm for weighted averaging, and is trained based on the numerical features created by the frequent term library in the sensitive metadata category, and the multi-classifier is constructed based on a multi-hidden layer neural network classifier with dropout parameters and L2 regularization, and feature compression and crossover are used to explore the interaction between features, and multi-classification is achieved through the softmax activation function.

[0011] In the metadata hierarchical classification method based on machine learning algorithm provided by the present invention, it can also have such technical features, wherein the predetermined algorithms include LR algorithm, multi-order crossover algorithm based on DeepFM and XGBoost algorithm, in the DeepFM algorithm, the Deep part uses dropout parameter and L2 regularization, the hidden layer activation function of the DeepFM algorithm is relu, the classification layer activation function uses sigmoid, the loss function uses cross entropy loss function, and F1_score is used as the early stopping evaluation index, the Adam optimizer is used to optimize the algorithm until the early stopping evaluation standard is reached, and the SGD optimizer is used to find the optimal point.

[0012] In the metadata hierarchical classification method based on machine learning algorithm provided by the present invention, it can also have such a technical feature, wherein the weighted average is to assign different weight coefficients according to the effects of each algorithm, and the weight coefficients are traversed and optimized by a for loop function to determine a set of weight coefficients that makes the F1_score of the Bagging model reach the maximum value.

[0013] The metadata hierarchical classification method based on the machine learning algorithm provided by the present invention may also have such a technical feature, wherein the weight coefficient of the two-stage classification model during training is:

[0014]

[0015] Where n class is the number of categories, N i is the number of financial metadata in each category.

[0016] Function and Effect of the Invention

[0017] According to the metadata hierarchical classification method based on machine learning algorithm of the present invention, a frequent term word library is first created according to the original metadata data set, without the need to pre-set the frequent term word library, thereby reducing the workload. Then, the features of the complex and rule-free text-like fields in the original metadata set are innovatively converted into numerical features based on the frequent term word library, thereby solving the problem that the existing text classification method cannot accurately classify metadata fields without unified naming rules. Secondly, a two-stage classification model with binary classifiers and multi-classifiers is constructed, and the model is trained and optimized using training data with extremely unbalanced classification. The metadata to be tested is first classified by sensitivity level by the binary classifier, and finally the sensitive metadata is further sub-classified by the multi-classifier to output a more accurate hierarchical classification result.

[0018] Through the metadata hierarchical classification method based on machine learning algorithm of the present invention, for complex and privacy-safe text-like metadata, a high-speed and high-quality hierarchical classification model can be used to perform accurate and reasonable hierarchical classification, achieve precise classification, effectively protect data privacy, meet business needs, and improve classification work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flowchart of a metadata classification method based on a machine learning algorithm in an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of the structure of a binary classifier in stage one of an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of the structure of a multi-classifier in stage 2 of an embodiment of the present invention;

[0022] Figure 4 1 is a schematic diagram comparing the binary classification effects of the Bagging ensemble model in stage 1 and other single classification algorithms in an embodiment of the present invention;

[0023] Figure 5 is a schematic diagram of classification effect of multiple classifiers in stage two of two-stage classification in an embodiment of the present invention; and

[0024] Figure 6 It is a comparison diagram of the classification effects of the non-stage classification model and the two-stage classification model in the embodiment of the present invention. DETAILED DESCRIPTION

[0025] The present invention proposes a metadata hierarchical classification method based on a machine learning algorithm, creates a sensitive frequent term library for metadata with text-like features, and on the basis of representing it as numerical features, integrates a two-stage machine learning algorithm to accurately classify the metadata.

[0026] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following is a detailed description of a metadata hierarchical classification method based on a machine learning algorithm of the present invention in combination with embodiments and drawings.

[0027] <Example>

[0028] Figure 1 It is a flowchart of a metadata classification method based on a machine learning algorithm in an embodiment of the present invention.

[0029] like Figure 1 As shown, the metadata classification method based on machine learning algorithm includes the following steps:

[0030] Step S1, constructing an original metadata set, and using stratified sampling to divide the original metadata set into a training set and a test set.

[0031] In this embodiment, financial metadata is collected from a structured database, and the table name field and column name field in the database are used as two columns of text features, namely Table_code and Column_code, respectively. An example is shown in Table 1 below.

[0032] Table_code Column_code Label TY_CREDIT_STOCK HOLDER_ACC_NO Account OPT_MONEY_OUT_APPLY CUST_NO Customer Number T_ZBMX_SMCPNRJBYLBDJE KHH Customer Number EXOUSER_TB PASSWORD password T_ZBMX_SMCPNRJBYLBDJE RYXM Name

[0033] Table 1

[0034] The labels corresponding to each data are digitally encoded to form 280458 original data samples, and these original data samples are classified into sensitive metadata categories and non-sensitive metadata categories according to the nature of the labels and stored as original metadata data sets.

[0035] Among them, samples with labels such as ID number, account, bank account number, customer number, password, name, address, phone number, email address, etc. belong to the sensitive metadata category. In this embodiment, there are 22,437 samples, and the remaining 258,021 samples belong to the non-sensitive metadata category. The number of samples in each label category is shown in Table 2 below.

[0036]

[0037] Table 2

[0038] In this embodiment, stratified sampling is used to take 75% of the total number of samples as the training set, which includes approximately 210,343 samples, and 15% of the total number of samples as the test set, which includes approximately 42,068 samples.

[0039] Step S2: Create a frequent term library based on the training set.

[0040] In this embodiment, according to the frequency of occurrence of the constituent fields, binary frequent items with a frequency greater than 20 times, ternary frequent items with a frequency greater than 20 times, and word frequent items with a frequency greater than 20 times or a frequency greater than 5 times in a certain type of sensitive data category are extracted from the original class text fields of Table_code and Column_code of the training set, and are merged into the Table_code frequent item vocabulary and the Column_code frequent item vocabulary, respectively.

[0041] Among them, bigram frequent items are language blocks represented by two letters, such as ID, IN, DU, etc.; trigram frequent items are language blocks represented by three letters, such as ACC, KHH, DDR, etc.; word frequent items are words separated by underscores, such as HOLDER, PASSWORD, CUST, etc.

[0042] Step S3: Based on the three frequent items in the frequent item lexicon, the text-like features of the original metadata are represented as numerical features.

[0043] Taking the Column_code field as an example, based on the Column_code frequent term database, a column of numerical features is created for each frequent term in the frequent term database, and the (0-1) numerical features corresponding to the sample are created based on whether the Column_code field contains the frequent term, that is, the text-like field features in the original metadata set are converted into (0-1) numerical features.

[0044] The value 0 indicates that the frequent item is not included in the Column_code field, and the value 1 indicates that the frequent item is included in the Column_code field. The creation of numerical features in the Table_code field is similar, and will not be described here. The specific creation example is shown in Table 3 below.

[0045] Table_code Column-code col_is_ID col_is_TEL col_is_ACC … tab_is_CREDIT CREDIT_STOCK ACCOUNT_ID 1 0 1 … 1

[0046] Table 3

[0047] Step S4, based on the extremely unbalanced original metadata sample, a two-stage classification model is constructed, which includes a binary classifier in stage one and a multi-classifier in stage two, and the two-stage classification model is trained using a training set represented by numerical features to obtain a hierarchical classification model.

[0048] Figure 2 is a schematic diagram of the structure of the binary classifier in stage one of the embodiment of the present invention and Figure 3 Schematic diagram of the structure of the multi-classifier in stage 2 of the embodiment of the present invention.

[0049] In this embodiment, the binary classifier in the first stage is constructed by using the weighted average bagging model integration method based on the LR (Logistic Regression) algorithm, the multi-order crossover algorithm based on DeepFM, and the XGBoost algorithm, and is trained using the numerical features created by the frequent word library in the sensitive metadata category. Figure 2 As shown, the binary classifier is used to calculate the classification probability of the metadata to be tested, and to make a large-category classification prediction based on the probability value.

[0050] The multi-classifier in the second stage is based on a multi-hidden layer neural network classifier with a dropout probability of 0.3 and an L2 regularization coefficient of 0.001 to prevent overfitting. It also uses feature compression and crossover to explore the interaction between features and realizes multi-classification through the softmax activation function. Figure 3 As shown, the multi-classifier is used to perform multi-classification on the results predicted by the binary classifier as the major category of sensitive data to obtain the corresponding sub-category results.

[0051] The Deep part of the DeepFM algorithm uses dropout with a probability of 0.3 and l2 regularization with a coefficient of 0.001. The hidden layer activation function uses relu, the classification layer activation function uses sigmoid, the loss function uses the cross entropy loss function, and F1_score is used as the early stopping evaluation index. The Adam optimizer with a learning rate of 0.001 is used for optimization until the early stopping evaluation index is reached, and then the SGD optimizer with a learning rate of 0.001 is used to find the optimal point.

[0052] The hyper parameters of the XGBoost algorithm are shown in Table 4 below:

[0053]

[0054]

[0055] Table 4

[0056] Figure 4 It is a schematic diagram comparing the binary classification effects of the Bagging ensemble model in stage one of an embodiment of the present invention and other single classification algorithms.

[0057] The weighted average of the Bagging ensemble model is to assign different weights according to the effect of each algorithm. Therefore, in this embodiment, the LR algorithm ( Figure 4 The classification effects of logistic regression, DeepFM algorithm, XGBoost algorithm and Bagging ensemble model were compared and tested. Figure 4 As shown in the figure, except for the Bagging ensemble model, the XGBoost algorithm has the best effect and is assigned a larger weight, while the LR algorithm has the worst effect and is assigned a smaller weight. The specific weight coefficients are determined by the for loop function through optimization to determine a set of weight coefficients that maximize the F1_score of the Bagging model.

[0058] In this embodiment, the training of the two-stage classification model is: inputting the metadata given in the training set into the two-stage classification model to predict the class label of the metadata. Specifically:

[0059] In the binary classifier of stage one, the largest number of non-sensitive metadata is classified as one category, and the remaining sensitive metadata is classified as one category, with the ratio of the two categories being 23:2. Due to data imbalance, the models of both stages are trained based on data plus weights, and are classified according to the calculated probability results: if the probability result calculated by the binary classifier is greater than or equal to 0.5, it is classified as the sensitive metadata category; if the calculated probability result is less than 0.5, it is classified as the non-sensitive metadata category.

[0060] The multi-classifier in stage two subdivides the samples predicted as sensitive data in the first stage into nine categories: bank account number, ID number, account number, customer number, password, name, address, telephone number, and email address.

[0061] In this embodiment, when the two-stage classification model is trained, the weight coefficients of each category are:

[0062]

[0063] Where n class is the number of categories, N i is the number of financial metadata in the training set in each category.

[0064] In order to verify the prediction effect of the trained hierarchical classification model, a test set containing 42,068 samples was used for verification. The classification effect of the hierarchical classification model on the sensitive data of the test set is shown in Table 5. As can be seen from Table 5, all nine types of sensitive data have a high F1-score and a low misjudgment rate.

[0065] precision recall fl-score support address 0.98 0.98 0.98 457 Name 0.98 0.97 0.98 569 Customer Number 0.96 0.95 0.95 705 password 1 1 1 62 Telephone 0.97 0.95 0.96 376 ID number 0.91 0.94 0.92 102 Account 0.87 0.9 0.88 306 Mail 1 0.98 0.99 128 Bank Account 0.77 0.88 0.82 52

[0066] Table 5

[0067] Step S5, inputting the metadata to be tested into the trained hierarchical classification model, and the model performs hierarchical prediction and classification calculation to obtain a hierarchical classification result of the metadata to be tested.

[0068] Figure 5 is a schematic diagram of the classification effect of the multi-classifier in the second stage of the two-stage classification in the embodiment of the present invention, and Figure 6 It is a comparison diagram of the classification effects of the non-stage classification model and the two-stage classification model in the embodiment of the present invention.

[0069] To further verify the effectiveness of the hierarchical classification model, the multi-classifier of the second stage is used to classify 9 types of sensitive data. The classification results are as follows: Figure 5 As shown in Figure 2, for comparison, a non-stage classification model is used to directly perform multi-classification on 10 types of samples. The classification results are shown in Figure 2. Figure 6 and as shown in Table 6 below.

[0070]

[0071]

[0072] Table 6

[0073] In summary, compared with the classification accuracy of the non-stage classification model, the accuracy of the two-stage classification of 9 types of sensitive data using the hierarchical classification model in this embodiment has been effectively improved. The address category has increased by 4%, the name category has increased by 26%, the customer number category has increased by 15%, the password category has increased by 14%, the telephone category has increased by 24%, the ID number category has increased by 38%, the account category has increased by 23%, the mailbox category has increased by 5%, and the bank account category has increased by 33%.

[0074] Example Function and Effect

[0075] According to a metadata classification method based on a machine learning algorithm provided in this embodiment, this method is aimed at complex and non-uniformly rule-free text-like metadata. It can convert the text-like metadata into data with numerical features on the basis of creating a frequent term library, and then integrate the two-stage machine learning algorithm to classify the metadata to be tested into sensitive levels and subclassify sensitive categories, thereby achieving accurate classification of metadata and solving the problem of huge cost caused by human labor in the current classification and classification of sensitive data in the financial field. At the same time, it also effectively protects data privacy, meets business needs, and improves classification efficiency.

[0076] In the embodiment, since a frequent word library is created based on the original metadata set, there is no need to pre-set the frequent word library, thus greatly reducing the workload of classification and grading. At the same time, based on the frequent word library, the features of the complex and rule-free quasi-text fields in the original metadata set are converted into numerical features, thereby solving the problem that the existing text classification method cannot accurately classify metadata fields without a unified naming rule.

[0077] In the embodiment, a two-stage classification model is constructed based on a machine learning algorithm, and the model is efficiently trained and tested using text data with numerical features. The metadata to be tested is first classified into sensitivity levels using a binary classifier, and then the sensitive metadata is further subclassified using a multi-classifier, thereby improving the accuracy of hierarchical classification.

[0078] The above embodiments are only used to illustrate specific implementation modes of the present invention, and the present invention is not limited to the description scope of the above embodiments.

Claims

1. A metadata classification method based on machine learning algorithm, It is characterized in that The following steps are involved: Step S1, constructing an original metadata set, and dividing the original metadata set into a training set and a test set by using stratified sampling; Step S2, creating a frequent term word library based on the training set; Step S3, representing the quasi-text features of the original metadata as numerical features based on the frequent items in the frequent item vocabulary; Step S4, constructing a two-stage classification model, and using a training set represented by numerical features to train the two-stage classification model to obtain a hierarchical classification model; Step S5, inputting the metadata to be tested into the hierarchical classification model, and using the hierarchical classification model to perform hierarchical prediction and classification calculation on the metadata to be tested to obtain corresponding hierarchical classification results. The two-stage classification model includes a binary classifier in stage one and a multi-classifier in stage two. The binary classifier is used to perform large-category classification prediction on the metadata to be tested. The multi-classifier performs multi-classification on the result of the hierarchical prediction to obtain corresponding sub-class results. The numerical feature is expressed as: For each frequent item, determine whether each class text field of the training set contains the frequent item, thereby creating a (0-1) numerical feature corresponding to the class text field. A value of 0 indicates that the text field does not contain the frequent item, and a value of 1 indicates that the text field contains the frequent item. The binary classifier is constructed by using a bagging model integration method based on a predetermined algorithm for weighted averaging, and is trained based on numerical features created from a frequent word library in a large category of sensitive metadata. The multi-classifier is constructed based on a multi-hidden layer neural network classifier with dropout parameters and L2 regularization, and uses feature compression and crossover to explore the interaction between features, and realizes the multi-classification through a softmax activation function.

2. The metadata classification method based on machine learning algorithm according to claim 1 is characterized in that: in, The construction process of the original metadata set is as follows: Collect financial metadata based on a structured database, where the financial metadata is a text-like field. The financial metadata with different labels are classified into sensitive metadata categories and non-sensitive metadata categories, so as to construct the original metadata data set, and the original metadata data set is divided into a training set and a test set by using stratified sampling.

3. The metadata classification method based on machine learning algorithm according to claim 2 is characterized by: in, The tags of financial metadata belonging to the broad category of sensitive metadata include at least account, bank account number, ID number, password, name, address, telephone number and email address.

4. The metadata classification method based on machine learning algorithm according to claim 2 is characterized in that: in, The creation process of the frequent term database is as follows: According to the frequency of occurrence of the constituent fields of the financial metadata, binary frequent items, ternary frequent items and single frequent items in the class text field are extracted from the training set, and a frequent item vocabulary corresponding to the class text field is obtained by merging these three frequent items.

5. The metadata classification method based on machine learning algorithm according to claim 1, characterized in that: in, The predetermined algorithms include LR algorithm, multi-order crossover algorithm based on DeepFM and XGBoost algorithm, The Deep part of the DeepFM algorithm uses the dropout parameter and L2 regularization. The hidden layer activation function of the DeepFM algorithm is relu, the classification layer activation function uses sigmoid, the loss function uses the cross entropy loss function, and F1_score is used as the early stopping evaluation index. The Adam optimizer is used to optimize the algorithm until the early stopping evaluation standard is met, and the SGD optimizer is used to find the optimal point.

6. The metadata classification method based on machine learning algorithm according to claim 1, characterized in that: in, The weighted average is to assign different weight coefficients according to the effect of each algorithm. The weight coefficients are traversed and optimized by a for loop function to determine a set of weight coefficients that maximize the F1_score of the Bagging model.

7. The metadata classification method based on machine learning algorithm according to claim 1 is characterized in that: in, The weight coefficient of the two-stage classification model during training is: Where n class is the number of categories, N i is the number of financial metadata in each category.