Database table classification method, device, readable storage medium and electronic device
Through the neural network model with multi-layer composite structure, the problem of lack of intelligence and relying on expert knowledge in the existing technology is solved, and intelligent grading of database tables in different industries and departments is realized, and grading efficiency and applicability are improved.
Patent Information
- Application Number
- CN202111679407.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-31
- Publication Date
- 2025-03-11
- Estimated Expiration
- 2041-12-31
AI Technical Summary
The existing rules-based database table grading method lacks intelligence and cannot be applied to grading tasks under different rules or standards. It relies on expert prior knowledge and artificial understanding of semantics, which is time-consuming and labor-intensive, and affects efficiency.
Using a neural network model with a multi-layer composite structure, by acquiring and constructing a training data set, the neural network model is trained to determine the confidentiality level of database tables, and intelligent grading of database tables in different industries and departments is achieved.
Intelligent rating of any database table from different industries and departments is realized, without the need for additional rule determination or training different models for different grading standards, improving the efficiency and universal applicability of library table grading.
Smart Images

Figure CN114297205B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of database table management, and in particular, to a database table classification method, device, readable storage medium and electronic device. Background Art
[0002] At present, database table classification is to determine the confidentiality level of the database table based on the source department of the database table, the industry it belongs to, and the sensitivity of the data it contains, thereby imposing different degrees of control on the opening and sharing of database tables of different confidentiality levels to achieve efficient management of data.
[0003] The mainstream technology for existing database table classification is the rule-based classification method. The rule-based classification method uses a set of "if...then..." rules to classify database tables: starting from the first "if...then..." rule, the current rule is used to match the content of the database table. If the current rule matches the content of the database table, the corresponding classification result under the current rule is used as the confidentiality level of the database table; if the current rule does not match the content of the database table, the next rule is used to match until a matching rule is found or the rules are exhausted, and the database table classification process ends. It is understandable that this rule-based classification method has one-to-one corresponding rule standards for database tables in different regions and fields, and relies on the prior knowledge of experts in the field. In the actual rule setting process, R&D personnel need to have some knowledge of the data itself and field knowledge, and after manually understanding the rules, the abstract laws contained therein are concretized into achievable functions.
[0004] Although the rule-based database table grading method has achieved intelligent grading to a certain extent, its implementation principle still lacks true intelligence. Its problems are mainly reflected in the following two aspects: First, the grading method can only be applied to grading tasks under the same rule or standard, and it is not reproducible and lacks generalization ability; second, the grading method relies on expert prior knowledge and human understanding of semantics, which is time-consuming and labor-intensive, thus affecting the efficiency of database table grading.
[0005] Therefore, how to provide a solution to the above technical problems is a problem that technicians in this field need to solve at present. Summary of the invention
[0006] The purpose of the present invention is to provide a database table classification method, device, readable storage medium and electronic device, which realizes intelligent classification of database tables by adopting a neural network model with a multi-layer composite structure. For any database table from different industries and departments, the corresponding confidentiality level can be directly output through the neural network model without any additional rule judgment and training different models for different classification standards, thereby realizing the generalization application of the model; moreover, the present invention performs semantic understanding of the information of the database table through the model itself, without the need for expert prior knowledge and human understanding of semantics, making the model more universally applicable and improving the efficiency of database table classification.
[0007] In order to achieve the above object, the present disclosure provides a database table classification method, comprising:
[0008] Obtain database table information of different industries and departments, and construct a training data set based on the database table information;
[0009] Constructing a neural network model of a multi-layer composite structure; wherein the neural network model is used to determine the confidentiality level of the database table according to the information of the database table;
[0010] Using the training data set to train the neural network model, to obtain a trained database table classification model;
[0011] A database table to be classified is obtained, and the database table to be classified is input into the database table classification model to obtain the confidentiality level of the database table to be classified.
[0012] Optionally, the database table information includes department names, table names, names of information items in the tables, and grading labels and industry labels corresponding to the different database tables.
[0013] Optionally, constructing a training data set based on the database table information includes:
[0014] Deduplication and removal of illegal information are performed on the department name, library table name and information item name in each database table;
[0015] Randomly adding or removing information items that have no effect on the grading result from the information items contained in each of the database tables;
[0016] Constructing new features for each information item contained in the database table; wherein the new features are feature variables associated with the information items in the table and having an impact on the classification results;
[0017] The newly constructed features corresponding to each of the database tables are combined with the department name, library table name and information item name in each of the database tables after final processing, as well as the classification label and industry label corresponding to each of the database tables to form the training data set.
[0018] Optionally, a neural network model with a multi-layer composite structure is constructed, including:
[0019] Constructing a first neural network structure; the first neural network structure is used to generate a semantic feature vector of a preset dimension based on the one-to-one correspondence between the department name, the table name and the information item name in the database table, and add the semantic feature vectors corresponding to the information item name in the same database table to obtain a semantic feature aggregation vector, and map the semantic feature aggregation vector to obtain a first feature vector with unchanged dimension;
[0020] Constructing a second neural network structure; the second neural network structure is used to combine the new features constructed corresponding to the same database table to obtain a combined feature, and generate a second feature vector with the same dimension as the semantic feature vector based on the combined feature;
[0021] Constructing a third neural network structure; the third neural network structure is used to classify the database table by industry based on the semantic feature vector corresponding to the department name of the database table;
[0022] Constructing a fourth neural network structure; the fourth neural network structure is used to assign preset vector weight values to the semantic feature vectors corresponding to the department name and the library table name of the same database table and the first feature vector and the second feature vector corresponding to the same database table, and add them to obtain a third feature vector, and based on the third feature vector, perform coarse-grained classification and fine-grained classification on the same database table; wherein the fine-grained classification corresponds to more classification types than the coarse-grained classification corresponds to the coarse-grained classification;
[0023] The first neural network structure, the second neural network structure, the third neural network structure and the fourth neural network structure are combined to obtain the neural network model of the multi-layer composite structure.
[0024] Optionally, constructing a first neural network structure includes:
[0025] Constructing an embedding layer; the embedding layer is used to map each character in the department name, library table name and information item name in the database table into a Chinese character vector;
[0026] Constructing a two-layer bidirectional recurrent neural network; the recurrent neural network is used to convert each of the Chinese character vectors into a semantic feature vector of a preset dimension;
[0027] Constructing a vector addition layer; the vector addition layer is used to add the semantic feature vectors corresponding to the information item names in the same database table to obtain a semantic feature aggregation vector;
[0028] Constructing a first fully connected layer; the first fully connected layer is used to map the semantic feature aggregation vector to obtain a semantic feature mapping vector with unchanged dimension;
[0029] Constructing a first dropout layer; the first dropout layer is used to map the semantic feature mapping vector to obtain a first feature vector with unchanged dimension.
[0030] Optionally, constructing a second neural network structure includes:
[0031] Constructing a second fully connected layer; the second fully connected layer is used to combine the new features constructed corresponding to the same database table to obtain combined features, and generate a new feature vector with the same dimension as the semantic feature vector based on the combined features;
[0032] Constructing a batch normalization layer; the batch normalization layer is used to perform numerical normalization processing on the new feature vector to obtain a new feature numerical normalized vector;
[0033] Constructing an activation function layer; the activation function layer is used to perform nonlinear processing on the new feature value normalized vector to obtain a new feature value normalized vector after nonlinear processing;
[0034] Construct a second dropout layer; the second dropout layer is used to map the normalized vector of the new feature value after nonlinear processing to obtain a second feature vector with unchanged dimension.
[0035] Optionally, construct a third neural network structure, including:
[0036] Constructing a third fully connected layer; the third fully connected layer is used to map the semantic feature vector corresponding to the department name in the database table to obtain a department feature mapping vector with unchanged dimension;
[0037] Constructing a first classification layer; the first classification layer is used to classify the database table by industry based on the department feature mapping vector.
[0038] Optionally, constructing a fourth neural network structure includes:
[0039] Constructing a vector weight addition layer; the vector weight addition layer is used to assign preset vector weight values to the semantic feature vectors corresponding to the department name and the library table name of the same database table and the first feature vector and the second feature vector corresponding to the same database table, and add them to obtain a third feature vector;
[0040] Constructing a fourth fully connected layer; the fourth fully connected layer is used to map the third feature vector to obtain a first mapping vector with unchanged dimension;
[0041] Constructing a second classification layer; the second classification layer is used to perform coarse-grained classification on the same database table based on the first mapping vector;
[0042] Constructing a fifth fully connected layer; the fifth fully connected layer is used to map the third feature vector to obtain a second mapping vector with unchanged dimension;
[0043] A third classification layer is constructed; the third classification layer is used to perform fine-grained classification on the same database table based on the second mapping vector.
[0044] Optionally, the neural network model is trained using the training data set to obtain a trained database table classification model, including:
[0045] Inputting information of different database tables in the training data set into the neural network model in sequence, and obtaining industry classification results, coarse-grained classification results and fine-grained classification results corresponding to the different database tables;
[0046] Substituting the industry classification result corresponding to the target database table and the industry label corresponding to the target database table into a preset loss function to calculate the loss, and obtaining a first loss; wherein the target database table is any database table in the training data set;
[0047] Substituting the coarse-grained classification result corresponding to the target database table and the coarse-grained classification label corresponding to the target database table into a preset loss function to perform loss calculation to obtain a second loss;
[0048] Substituting the fine-grained classification result corresponding to the target database table and the fine-grained classification label corresponding to the target database table into a preset loss function to perform loss calculation to obtain a third loss;
[0049] Assigning a preset loss weight value to the first loss, the second loss, and the third loss and adding them together to obtain a total loss;
[0050] With the optimization goal of reducing the total loss to 0, the parameters of the neural network model are optimized and adjusted using a preset neural network model optimization algorithm until all the training data sets are trained on the neural network model to obtain a trained database table classification model.
[0051] In order to achieve the above object, the present disclosure also provides a database table grading device, comprising:
[0052] A data construction unit, used to obtain database table information of different industries and departments, and to construct a training data set based on the database table information;
[0053] A model building unit, used to build a neural network model of a multi-layer composite structure; wherein the neural network model is used to determine the confidentiality level of the database table according to the information of the database table;
[0054] A model training unit, used to train the neural network model using the training data set to obtain a trained database table classification model;
[0055] The database table grading unit is used to obtain a database table to be graded, and input the database table to be graded into the database table grading model to obtain the confidentiality level of the database table to be graded.
[0056] In order to achieve the above-mentioned purpose, the present disclosure further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned database table grading methods are implemented.
[0057] In order to achieve the above object, the present disclosure further provides an electronic device, including:
[0058] a memory having a computer program stored thereon;
[0059] A processor is used to execute the computer program in the memory to implement the steps of any one of the above-mentioned database table grading methods.
[0060] Through the above technical solution, the present disclosure adopts a neural network model with a multi-layer composite structure to realize intelligent classification of database tables. For any database table from different industries and departments, the corresponding confidentiality level can be directly output through the neural network model without any additional rule judgment and training different models for different classification standards, thereby realizing the generalization application of the model; moreover, the present disclosure performs semantic understanding of the information of the database table through the model itself, without the need for expert prior knowledge and human understanding of semantics, making the model more universally applicable and improving the efficiency of database table classification.
[0061] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the prior art and the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0063] Figure 1 A flowchart of a database table classification method provided in an embodiment of the present disclosure;
[0064] Figure 2 A classification principle diagram of a neural network model of a multi-layer composite structure provided in an embodiment of the present disclosure;
[0065] Figure 3 A schematic diagram of the structure of a database table grading device provided in an embodiment of the present disclosure;
[0066] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;
[0067] Figure 5 A schematic diagram of the structure of another electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0068] The core of the present disclosure is to provide a database table classification method, device, readable storage medium and electronic device, which realizes intelligent classification of database tables by adopting a neural network model with a multi-layer composite structure. For any database table from different industries and departments, the corresponding confidentiality level can be directly output through the neural network model without any additional rule judgment and training different models for different classification standards, thereby realizing the generalization application of the model; moreover, the present disclosure performs semantic understanding of the information of the database table through the model itself, without the need for expert prior knowledge and human understanding of semantics, making the model more universally applicable and improving the efficiency of database table classification.
[0069] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0070] Please refer to Figure 1 , Figure 1 A flowchart of a database table grading method provided in an embodiment of the present disclosure.
[0071] The database table classification method includes:
[0072] Step 101: Obtain database table information of different industries and departments, and build a training data set based on the database table information.
[0073] Specifically, considering that database table classification determines the confidentiality level of a database table based on the source department of the database table, the industry to which it belongs, and the sensitivity of the data contained, the present invention obtains database table information of different industries and departments to construct a training data set based on the database table information (for use in training a neural network model for database table classification).
[0074] Step 102: construct a neural network model with a multi-layer composite structure; wherein the neural network model is used to determine the confidentiality level of the database table according to the information in the database table.
[0075] Specifically, the present invention constructs a neural network model of a multi-layer composite structure. The purpose of constructing the neural network model is to be applied to database table classification, that is, the information of a database table is input into the neural network model, and the neural network model can output the confidentiality level of this database table.
[0076] Step 103: Use the training data set to train the neural network model to obtain a trained database table classification model.
[0077] Specifically, the present disclosure utilizes a training data set to train a neural network model, with the goal of improving the accuracy of model classification by training the neural network model, and ultimately obtaining a database table classification model with higher accuracy.
[0078] Step 104: Obtain the database table to be classified, and input the database table to be classified into the database table classification model to obtain the confidentiality level of the database table to be classified.
[0079] Specifically, after obtaining a database table classification model with high accuracy, the present disclosure can obtain a database table to be classified, and then input the database table to be classified into the database table classification model to obtain the confidentiality level of the database table to be classified.
[0080] It can be seen that the present disclosure adopts a neural network model with a multi-layer composite structure to realize the intelligent classification of database tables. For any database table from different industries and departments, the corresponding confidentiality level can be directly output through the neural network model without any additional rule judgment and training different models for different classification standards, thus realizing the generalization application of the model; moreover, the present disclosure performs semantic understanding of the information of the database table through the model itself, without the need for expert prior knowledge and human understanding of semantics, making the model more universally applicable and improving the efficiency of database table classification.
[0081] Based on the above embodiments:
[0082] As an optional embodiment, the database table information includes department names, table names, names of information items in the tables, and hierarchical labels and industry labels corresponding to the different database tables.
[0083] Specifically, for the database table information of different industries and departments, the information of each database table may include the department name, table name, and information item name in the table (the information item name refers to the name of each type of data in the database table. For example, if a database table contains data such as name, unit, medical history, etc., then "name", "unit", "medical history", etc. are the information item names of the database table). The information of each database table may also include the classification label corresponding to the database table (indicating the actual confidentiality level corresponding to the database table) and the industry label (indicating the industry in which the database table actually belongs).
[0084] As an optional embodiment, constructing a training data set based on database table information includes:
[0085] The department name, table name and information item name in each database table are deduplicated and illegal information is removed;
[0086] Randomly add or remove information items that have no effect on the classification results from the information items contained in each database table;
[0087] Constructing new features for each information item contained in the database table; wherein the new features are feature variables associated with the information items in the table and having an impact on the classification results;
[0088] The newly constructed features corresponding to each database table are combined with the department name, library table name and information item name of each database table after final processing, as well as the hierarchical label and industry label corresponding to each database table to form a training data set.
[0089] Specifically, the process of constructing a training data set based on database table information of different industries and departments includes: 1) pre-processing the department name, library table name and information item name in each database table, including deduplication (removing duplicate department names, library table names and information item names in the table. For example, if the information item name of the database table contains two "name" information items, deduplication means removing a duplicate "name" information item) and removing illegal information (such as English information). 2) Data enhancement processing is performed on the information items contained in each database table, that is, randomly adding or removing information items that have no effect on the classification results in the information items contained in each database table. The effect is reflected in: when the data of the training data set is limited, by performing data enhancement processing on the existing data, the risk of overfitting of the model is reduced, and the robustness and accuracy of the model are improved. 3) Construct new features for the information items contained in each database table. The new features are feature variables associated with the information items in the table and have an impact on the grading results. Multiple Boolean feature variables can be constructed as new features. For example, one of the new features constructed for the information items contained in a database table indicates whether the information items contained in the database table contain an ID number. If the ID number is contained, the new feature is assigned a value of 1. If the ID number is not contained, the new feature is assigned a value of 0. By analogy, multiple new features are constructed and assigned according to actual needs. It can be understood that these constructed new features further mine the information that affects the grading results in addition to the text information of the information items contained in the database table, thereby improving the model effect. 4) Combine the new features constructed corresponding to each database table with the department name, library table name, and information item name in the table after the final processing of each database table (after a series of processing such as deduplication, removal of illegal information, and data enhancement), as well as the grading label and industry label corresponding to each database table to form a training data set.
[0090] As an optional embodiment, constructing a neural network model with a multi-layer composite structure includes:
[0091] Constructing a first neural network structure; the first neural network structure is used to generate a semantic feature vector of a preset dimension based on the one-to-one correspondence between the department name, the table name and the information item name in the database table, and add the semantic feature vectors corresponding to the information item name in the same database table to obtain a semantic feature aggregation vector, and map the semantic feature aggregation vector to obtain a first feature vector with unchanged dimension;
[0092] Constructing a second neural network structure; the second neural network structure is used to combine the new features corresponding to the same database table to obtain a combined feature, and generate a second feature vector with the same dimension as the semantic feature vector based on the combined feature;
[0093] Constructing a third neural network structure; the third neural network structure is used to classify the database table by industry based on the semantic feature vector corresponding to the department name of the database table;
[0094] Constructing a fourth neural network structure; the fourth neural network structure is used to assign preset vector weight values to the semantic feature vectors corresponding to the department name and the library table name of the same database table and the first feature vector and the second feature vector corresponding to the same database table, and add them to obtain a third feature vector, and based on the third feature vector, perform coarse-grained classification and fine-grained classification on the same database table; wherein the fine-grained classification corresponds to more classification types than the coarse-grained classification corresponds to the classification types;
[0095] The first neural network structure, the second neural network structure, the third neural network structure and the fourth neural network structure are combined to obtain a neural network model with a multi-layer composite structure.
[0096] Specifically, the process of constructing a neural network model with a multi-layer composite structure for database table classification includes: 1) constructing a first neural network structure; the purpose of constructing the first neural network structure is to generate a semantic feature vector of a preset dimension (such as 256 dimensions) based on the department name, the table name and the information item name in the database table. Since there is no sequence relationship between all the information item names contained in a single database table, the semantic feature vectors corresponding to the information item names in the same database table are added to realize the disordered aggregation of the information items, and finally a single semantic feature aggregation vector is obtained, and then the semantic feature aggregation vector is mapped to obtain a first feature vector with unchanged dimension, that is, the first feature vector has the same dimension as the semantic feature aggregation vector. The purpose of mapping is to increase the model complexity and improve the fit. 2) Constructing a second neural network structure; the purpose of constructing the second neural network structure is to combine the new features corresponding to the same database table to obtain a combined feature, and then generate a second feature vector with the same dimension as the semantic feature vector based on the combined feature. 3) Constructing a third neural network structure; the purpose of constructing the third neural network structure is to classify the database table by industry based on the semantic feature vector corresponding to the department name of the database table. 4) Construct the fourth neural network structure; the purpose of constructing the fourth neural network structure is to assign preset vector weight values to the semantic feature vectors corresponding to the department name and the table name of any database table, as well as the first feature vector and the second feature vector corresponding to the database table, and add the semantic feature vector, the first feature vector and the second feature vector corresponding to the database table with preset vector weight values to obtain the third feature vector, and then perform coarse-grained classification and fine-grained classification on the database table based on the third feature vector; it should be noted that the classification types corresponding to the fine-grained classification are more than the classification types corresponding to the coarse-grained classification, for example, under the fine-grained classification, the database table is divided into four levels: 1, 2, 3, and 4; under the coarse-grained classification, the database table is divided into two levels: high and low (levels 1 and 2 belong to the high level, and levels 3 and 4 belong to the low level). 5) By combining the first neural network structure, the second neural network structure, the third neural network structure and the fourth neural network structure, a neural network model with a multi-layer composite structure can be obtained, which is used to realize database table classification.
[0097] Please refer to Figure 2 , Figure 2 A classification principle diagram of a neural network model with a multi-layer composite structure provided in an embodiment of the present disclosure.
[0098] As an optional embodiment, constructing a first neural network structure includes:
[0099] Construct an embedding layer; the embedding layer is used to map each character in the department name, library table name, and information item name based on the database table into a Chinese character vector;
[0100] Construct a two-layer bidirectional recurrent neural network; the recurrent neural network is used to convert each Chinese character vector into a semantic feature vector of a preset dimension;
[0101] Constructing a vector addition layer; the vector addition layer is used to add the semantic feature vectors corresponding to the information item names in the same database table to obtain a semantic feature aggregation vector;
[0102] Constructing a first fully connected layer; the first fully connected layer is used to map the semantic feature aggregation vector to obtain a semantic feature mapping vector with unchanged dimension;
[0103] Construct the first dropout layer; the first dropout layer is used to map the semantic feature mapping vector to obtain a first feature vector with unchanged dimension.
[0104] Specifically, the process of constructing the first neural network structure includes: 1) constructing an embedding layer (embedding literally means "embedding", which is essentially a mapping from semantic space to vector space, while maintaining the relationship between the original sample characters in the semantic space in the vector space as much as possible, such as the positions of two characters with similar semantics in the vector space are also relatively close); the purpose of constructing the embedding layer is to map each character in the department name, library table name and information item name based on the database table into a Chinese character vector (such as a 300-dimensional Chinese character vector) to obtain text literal information; wherein, the embedding layer can use the word2vec model to achieve mapping. 2) Constructing a two-layer bidirectional recurrent neural network; the purpose of constructing the recurrent neural network is to convert each Chinese character vector into a semantic feature vector of a preset dimension (such as a 256-dimensional semantic feature vector), that is, to use the two-layer bidirectional recurrent neural network as a basic neuron to extract text semantic features. It should be noted that recurrent neural networks are widely used in the field of natural language processing. In consideration of the computational efficiency in practical applications, the present disclosure adopts a two-layer bidirectional GRU (Gate Recurrent Unit) structure for the two-layer bidirectional recurrent neural network. The purpose is to obtain the semantic information of the text with a deeper network structure on the basis of obtaining the literal information of the text. 3) Construct a vector addition layer; the purpose of constructing the vector addition layer is to add the semantic feature vectors corresponding to the information item names in the same database table to obtain the semantic feature aggregation vector. 4) Construct a first fully connected layer; the purpose of constructing the first fully connected layer is to map the semantic feature aggregation vector to obtain a semantic feature mapping vector with unchanged dimension, that is, the dimension of the semantic feature mapping vector is the same as that of the semantic feature aggregation vector. 5) Construct a first dropout (remove some neurons) layer; the purpose of constructing the first dropout layer is to map the semantic feature mapping vector to obtain a first feature vector with unchanged dimension, that is, the dimension of the first feature vector is the same as that of the semantic feature mapping vector. The function of the first fully connected layer is to map the vector to another space, increase the complexity of the model, and improve the fit. The function of the first dropout layer is to prevent the model from overfitting.
[0105] The first neural network is a combination of an embedding layer, a two-layer bidirectional recurrent neural network, a vector addition layer, a first fully connected layer, and a first dropout layer.
[0106] As an optional embodiment, constructing a second neural network structure includes:
[0107] Constructing a second fully connected layer; the second fully connected layer is used to combine the new features constructed corresponding to the same database table to obtain combined features, and generate a new feature vector with the same dimension as the semantic feature vector based on the combined features;
[0108] Construct a batch normalization layer; the batch normalization layer is used to perform numerical normalization on the new feature vector to obtain a new feature numerical normalized vector;
[0109] Constructing an activation function layer; the activation function layer is used to perform nonlinear processing on the normalized vector of new feature values to obtain a normalized vector of new feature values after nonlinear processing;
[0110] Construct a second dropout layer; the second dropout layer is used to map the normalized vector of the new feature value after nonlinear processing to obtain a second feature vector with unchanged dimension.
[0111] Specifically, the process of constructing the second neural network structure includes: 1) constructing the second fully connected layer; the purpose of constructing the second fully connected layer is to combine the new features corresponding to the same database table to obtain combined features, and then generate a new feature vector with the same dimension as the semantic feature vector based on the combined features. 2) Constructing the BN (batch normalization) layer; the purpose of constructing the batch normalization layer is to perform numerical normalization on the new feature vector, that is, to map the values in the new feature vector to the range of [0,1] to obtain a new feature numerical normalized vector for subsequent processing. 3) Constructing the activation function layer; the purpose of constructing the activation function layer is to perform nonlinear processing on the new feature numerical normalized vector to obtain a new feature numerical normalized vector after nonlinear processing; wherein, the activation function layer can be implemented using the ReLU (Rectified LinearUnit) function. 4) Construct the second dropout layer; the purpose of constructing the second dropout layer is to map the new feature value normalized vector after nonlinear processing to obtain a second feature vector with unchanged dimension, that is, the second feature vector has the same dimension as the new feature level normalized vector. The role of the second fully connected layer is to map the vector to another space, increase the complexity of the model, and improve the fit. The role of the second dropout layer is to prevent the model from overfitting.
[0112] In addition, after obtaining the new feature value normalized vector after nonlinear processing, the activation function layer can also convert each value of the new feature value normalized vector after nonlinear processing into values of the same numerical magnitude, and then input it into the second dropout layer. For example, the value of the new feature value normalized vector after nonlinear processing may be 0.05 (numerical magnitude is 0.01) or 0.9 (numerical magnitude is 0.1). Each value of the new feature value normalized vector after nonlinear processing can be converted into a value of 0.01 numerical magnitude, which is more convenient for subsequent processing.
[0113] The second neural network is the combination of the second fully connected layer, batch normalization layer, activation function layer and the second dropout layer.
[0114] As an optional embodiment, constructing a third neural network structure includes:
[0115] Constructing a third fully connected layer; the third fully connected layer is used to map the semantic feature vector corresponding to the department name in the database table to obtain a department feature mapping vector with unchanged dimension;
[0116] Construct the first classification layer; the first classification layer is used to classify the database table by industry based on the department feature mapping vector.
[0117] Specifically, the process of constructing the third neural network structure includes: 1) constructing the third fully connected layer; the purpose of constructing the third fully connected layer is to map the semantic feature vector corresponding to the department name of the database table to obtain a department feature mapping vector with unchanged dimension, that is, the dimension of the department feature mapping vector is the same as that of the semantic feature vector. 2) constructing the first classification layer (implemented with the help of the softmax function); the purpose of constructing the first classification layer is to classify the database table by industry based on the department feature mapping vector corresponding to a database table, so as to calculate the loss when the model is applied.
[0118] Then the third neural network is the combination of the third fully connected layer and the first classification layer.
[0119] As an optional embodiment, constructing a fourth neural network structure includes:
[0120] Constructing a vector weight addition layer; the vector weight addition layer is used to assign preset vector weight values to the semantic feature vectors corresponding to the department name and the library table name of the same database table and the first feature vector and the second feature vector corresponding to the same database table, and add them to obtain a third feature vector;
[0121] Constructing a fourth fully connected layer; the fourth fully connected layer is used to map the third eigenvector to obtain a first mapping vector with unchanged dimension;
[0122] Constructing a second classification layer; the second classification layer is used to perform coarse-grained classification on the same database table based on the first mapping vector;
[0123] Constructing a fifth fully connected layer; the fifth fully connected layer is used to map the third feature vector to obtain a second mapping vector with unchanged dimension;
[0124] A third classification layer is constructed; the third classification layer is used to perform fine-grained classification on the same database table based on the second mapping vector.
[0125] Specifically, the process of constructing the fourth neural network structure includes: 1) constructing a vector weight addition layer; the purpose of constructing the vector weight addition layer is to assign preset vector weight values to the semantic feature vectors corresponding to the department name and the table name of any database table and the first feature vector and the second feature vector corresponding to the database table, and add the semantic feature vector, the first feature vector and the second feature vector corresponding to the database table with preset vector weight values to obtain the third feature vector. 2) Constructing the fourth fully connected layer; the purpose of constructing the fourth fully connected layer is to map the third feature vector to obtain a first mapping vector with unchanged dimension, that is, the first mapping vector has the same dimension as the third feature vector. 3) Constructing the second classification layer (implemented with the help of the softmax function); the purpose of constructing the second classification layer is to perform coarse-grained classification of a database table based on the first mapping vector corresponding to the database table. 4) Constructing the fifth fully connected layer; the purpose of constructing the fifth fully connected layer is to map the third feature vector to obtain a second mapping vector with unchanged dimension, that is, the second mapping vector has the same dimension as the third feature vector. 5) Constructing a third classification layer (implemented with the help of a softmax function); the purpose of constructing the third classification layer is to perform fine-grained classification on a database table based on the second mapping vector corresponding to the database table.
[0126] Then the fourth neural network is a combination of the vector weight addition layer, the fourth fully connected layer, the second classification layer, the fifth fully connected layer and the third classification layer.
[0127] As an optional embodiment, the neural network model is trained using the training data set to obtain a trained database table classification model, including:
[0128] The information of different database tables in the training data set is input into the neural network model in sequence to obtain the industry classification results, coarse-grained classification results and fine-grained classification results corresponding to different database tables;
[0129] Substituting the industry classification result corresponding to the target database table and the industry label corresponding to the target database table into the preset loss function to calculate the loss, and obtaining the first loss; wherein the target database table is any database table in the training data set;
[0130] Substituting the coarse-grained classification result corresponding to the target database table and the coarse-grained classification label corresponding to the target database table into a preset loss function to calculate the loss, thereby obtaining a second loss;
[0131] Substituting the fine-grained classification result corresponding to the target database table and the fine-grained classification label corresponding to the target database table into a preset loss function to calculate the loss, thereby obtaining a third loss;
[0132] Assigning preset loss weight values to the first loss, the second loss, and the third loss and adding them together to obtain the total loss;
[0133] With the optimization goal of reducing the total loss to 0, the parameters of the neural network model are optimized and adjusted using the preset neural network model optimization algorithm until all training data sets are trained on the neural network model to obtain a trained database table classification model.
[0134] Specifically, the training process of the neural network model includes: 1) inputting the information of different database tables in the training data set into the neural network model in sequence (the above-mentioned embodiment of constructing the neural network model has explained the model principle in detail, and the present disclosure will not repeat it here), and the industry classification results, coarse-grained grading results and fine-grained grading results corresponding to different database tables can be obtained. 2) Take the target database table for illustration. The industry classification result corresponding to the target database table is the industry of the target database table predicted by the neural network model. The industry label corresponding to the target database table is the industry actually located in the target database table. The industry classification result corresponding to the target database table and the industry label corresponding to the target database table can be substituted into a preset loss function (such as a cross entropy loss function) for loss calculation to obtain a first loss Loss1; the grading label corresponding to the target database table includes a coarse-grained grading label and a fine-grained grading label; the coarse-grained grading result corresponding to the target database table is the coarse-grained confidentiality level corresponding to the target database table predicted by the neural network model. The coarse-grained grading label corresponding to the target database table is the coarse-grained confidentiality level actually corresponding to the target database table. The coarse-grained grading result corresponding to the target database table and the coarse-grained grading label corresponding to the target database table can be substituted into a preset loss function (such as a cross entropy loss function) for loss calculation to obtain a second loss Loss2; the fine-grained grading result corresponding to the target database table is the fine-grained confidentiality level corresponding to the target database table predicted by the neural network model. , the fine-grained classification label corresponding to the target database table is the fine-grained confidentiality level actually corresponding to the target database table. The fine-grained classification result corresponding to the target database table and the fine-grained classification label corresponding to the target database table can be substituted into the preset loss function (such as the cross entropy loss function) for loss calculation to obtain the third loss Loss3. 3) Assign a loss weight value λ1 to the first loss Loss1, a loss weight value λ2 to the second loss Loss2, and a loss weight value λ3 to the third loss Loss3, and add the first loss, the second loss and the third loss assigned with the loss weight value to obtain the total loss Loss = λ1*Loss1+λ2*Loss2+λ3*Loss3. 4) With the optimization goal of reducing the total loss Loss to 0 (that is, the optimization goal is to make the prediction result of the neural network model as consistent as possible with the actual situation of the database table), the preset neural network model optimization algorithm is used to optimize and adjust the parameters of the neural network model until all the training data sets are trained on the neural network model, and a trained database table classification model is obtained, and the neural network model training is completed.Among them, the neural network model optimization algorithm can adopt the Adam (Adaptive moment estimation) algorithm. For the Adam algorithm, its parameters can be set as: the learning rate is set to 0.001, the weight ratio of the three losses is set to λ1:λ2:λ3=1:4:80, the weight decay is set to 0.00005, and the batch size is set to 64 (it can also be set to other values according to actual conditions).
[0135] It can be seen that under the deep learning framework, multi-task output is achieved through forward propagation of the same model, and the coarse-grained classification results, fine-grained classification results and industry classification results corresponding to the database table are output at the same time. There is no need to train multiple models for different output tasks.
[0136] In summary, in the intelligent grading process disclosed in the present invention, semantic understanding of the information item text data of the database table itself, as well as the department name and table name text data corresponding to the database table is achieved, and parameters are adjusted according to the importance of different text data, and the network structure is optimized to improve the accuracy of the intelligent grading results. The effect is reflected in: inputting the department name, library table name and all information item names contained in each database table, realizing the analysis and understanding of the text semantics by the neural network, learning the grading rules contained therein, and finally completing the intelligent grading task with high accuracy through the layer-by-layer transmission of the neural network.
[0137] Please refer to Figure 3 , Figure 3 A schematic diagram of the structure of a database table grading device provided in an embodiment of the present disclosure.
[0138] The database table grading device 100 comprises:
[0139] The data construction unit 101 is used to obtain database table information of different industries and departments, and to construct a training data set based on the database table information;
[0140] The model building unit 102 is used to build a neural network model of a multi-layer composite structure; wherein the neural network model is used to determine the confidentiality level of the database table according to the information in the database table;
[0141] The model training unit 103 is used to train the neural network model using the training data set to obtain a trained database table classification model;
[0142] The database table classification unit 104 is used to obtain the database table to be classified, and input the database table to be classified into the database table classification model to obtain the confidentiality level of the database table to be classified.
[0143] As an optional embodiment, the database table information includes department names, table names, names of information items in the tables, and hierarchical labels and industry labels corresponding to the different database tables.
[0144] As an optional embodiment, the data construction unit 101 includes:
[0145] The pre-processing module is used to remove duplicate and illegal information from the department name, library table name and information item name in each database table;
[0146] A data enhancement module is used to randomly add or remove information items that have no effect on the classification results from the information items contained in each database table;
[0147] A feature construction module is used to construct new features for each information item contained in each database table; wherein the new features are feature variables associated with the information items in the table and have an impact on the classification results;
[0148] The information combination module is used to combine the newly constructed features corresponding to each database table with the department name, library table name and information item name in each database table after final processing, as well as the hierarchical label and industry label corresponding to each database table to form a training data set.
[0149] As an optional embodiment, the model building unit 102 includes:
[0150] The first construction module is used to construct a first neural network structure; the first neural network structure is used to generate a semantic feature vector of a preset dimension based on the one-to-one correspondence between the department name, the table name and the information item name in the database table, and add the semantic feature vectors corresponding to the information item name in the same database table to obtain a semantic feature aggregation vector, and map the semantic feature aggregation vector to obtain a first feature vector with unchanged dimension;
[0151] A second construction module is used to construct a second neural network structure; the second neural network structure is used to combine the new features corresponding to the same database table to obtain a combined feature, and generate a second feature vector with the same dimension as the semantic feature vector based on the combined feature;
[0152] A third building module is used to build a third neural network structure; the third neural network structure is used to classify the database table by industry based on the semantic feature vector corresponding to the department name of the database table;
[0153] The fourth construction module is used to construct a fourth neural network structure; the fourth neural network structure is used to assign preset vector weight values to the semantic feature vectors corresponding to the department name and the library table name of the same database table and the first feature vector and the second feature vector corresponding to the same database table, and add them to obtain a third feature vector, and perform coarse-grained classification and fine-grained classification on the same database table based on the third feature vector; wherein the classification types corresponding to the fine-grained classification are more than the classification types corresponding to the coarse-grained classification;
[0154] The structure combination module is used to combine the first neural network structure, the second neural network structure, the third neural network structure and the fourth neural network structure to obtain a neural network model with a multi-layer composite structure.
[0155] As an optional embodiment, the first building block includes:
[0156] The first construction submodule is used to construct an embedding layer; the embedding layer is used to map each character in the department name, library table name and information item name in the database table into a Chinese character vector;
[0157] The second construction submodule is used to construct a two-layer bidirectional recurrent neural network; the recurrent neural network is used to convert each Chinese character vector into a semantic feature vector of a preset dimension;
[0158] The third construction submodule is used to construct a vector addition layer; the vector addition layer is used to add the semantic feature vectors corresponding to the information item names in the same database table to obtain a semantic feature aggregation vector;
[0159] The fourth construction submodule is used to construct a first fully connected layer; the first fully connected layer is used to map the semantic feature aggregation vector to obtain a semantic feature mapping vector with unchanged dimension;
[0160] The fifth construction submodule is used to construct a first dropout layer; the first dropout layer is used to map the semantic feature mapping vector to obtain a first feature vector with unchanged dimension.
[0161] As an optional embodiment, the second building block includes:
[0162] The sixth construction submodule is used to construct a second fully connected layer; the second fully connected layer is used to combine the new features constructed corresponding to the same database table to obtain combined features, and generate a new feature vector with the same dimension as the semantic feature vector based on the combined features;
[0163] The seventh construction submodule is used to construct a batch normalization layer; the batch normalization layer is used to perform numerical normalization processing on the new feature vector to obtain a new feature numerical normalized vector;
[0164] The eighth construction submodule is used to construct an activation function layer; the activation function layer is used to perform nonlinear processing on the new feature value normalized vector to obtain a new feature value normalized vector after nonlinear processing;
[0165] The ninth construction submodule is used to construct a second dropout layer; the second dropout layer is used to map the normalized vector of the new feature value after nonlinear processing to obtain a second feature vector with unchanged dimension.
[0166] As an optional embodiment, the third building block includes:
[0167] The tenth construction submodule is used to construct a third fully connected layer; the third fully connected layer is used to map the semantic feature vector corresponding to the department name in the database table to obtain a department feature mapping vector with unchanged dimension;
[0168] The eleventh construction submodule is used to construct a first classification layer; the first classification layer is used to classify the database table by industry based on the department feature mapping vector.
[0169] As an optional embodiment, the fourth building block includes:
[0170] The twelfth construction submodule is used to construct a vector weight addition layer; the vector weight addition layer is used to assign a preset vector weight value to the semantic feature vector corresponding to the department name and the library table name of the same database table and the first feature vector and the second feature vector corresponding to the same database table, and add them to obtain a third feature vector;
[0171] A thirteenth construction submodule is used to construct a fourth fully connected layer; the fourth fully connected layer is used to map the third feature vector to obtain a first mapping vector with unchanged dimension;
[0172] A fourteenth construction submodule is used to construct a second classification layer; the second classification layer is used to perform coarse-grained classification on the same database table based on the first mapping vector;
[0173] A fifteenth construction submodule is used to construct a fifth fully connected layer; the fifth fully connected layer is used to map the third feature vector to obtain a second mapping vector with unchanged dimension;
[0174] The sixteenth construction submodule is used to construct a third classification layer; the third classification layer is used to perform fine-grained classification on the same database table based on the second mapping vector.
[0175] As an optional embodiment, the model training unit 103 includes:
[0176] The data input module is used to input the information of different database tables in the training data set into the neural network model in sequence to obtain the industry classification results, coarse-grained classification results and fine-grained classification results corresponding to different database tables;
[0177] A first loss calculation module is used to substitute the industry classification result corresponding to the target database table and the industry label corresponding to the target database table into a preset loss function to perform loss calculation to obtain a first loss; wherein the target database table is any database table in the training data set;
[0178] A second loss calculation module is used to substitute the coarse-grained classification result corresponding to the target database table and the coarse-grained classification label corresponding to the target database table into a preset loss function to perform loss calculation to obtain a second loss;
[0179] A third loss calculation module is used to substitute the fine-grained classification result corresponding to the target database table and the fine-grained classification label corresponding to the target database table into a preset loss function to perform loss calculation to obtain a third loss;
[0180] A loss summing module, used to assign preset loss weight values to the first loss, the second loss and the third loss and add them together to obtain a total loss;
[0181] The model optimization module is used to optimize and adjust the parameters of the neural network model by using a preset neural network model optimization algorithm with the optimization goal of reducing the total loss to 0, until all training data sets are trained on the neural network model, and a trained database table classification model is obtained.
[0182] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0183] Figure 4 FIG. 2 is a block diagram of an electronic device 200 according to an exemplary embodiment. Figure 4 As shown, the electronic device 200 may include: a processor 201 and a memory 202. The electronic device 200 may also include one or more of a multimedia component 203, an input / output (I / O) interface 204, and a communication component 205.
[0184] The processor 201 is used to control the overall operation of the electronic device 200 to complete all or part of the steps in the above-mentioned database table classification method. The memory 202 is used to store various types of data to support the operation of the electronic device 200, and these data may include, for example, instructions for any application or method used to operate on the electronic device 200, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 202 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 203 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the memory 202 or sent through the communication component 205. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 204 provides an interface between the processor 201 and other interface modules, and the above-mentioned other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 205 is used for wired or wireless communication between the electronic device 200 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 205 may include: Wi-Fi module, Bluetooth module, NFC module.
[0185] In an exemplary embodiment, the electronic device 200 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned database table classification method.
[0186] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned database table classification method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 202 including program instructions, and the above-mentioned program instructions can be executed by the processor 201 of the electronic device 200 to complete the above-mentioned database table classification method.
[0187] Figure 5 FIG. 1 is a block diagram of another electronic device 300 according to an exemplary embodiment. For example, the electronic device 300 may be provided as a server. Figure 5 The electronic device 300 includes a processor 301, which may be one or more, and a memory 303 for storing a computer program executable by the processor 301. The computer program stored in the memory 303 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 301 may be configured to execute the computer program to perform the above-mentioned database table classification method.
[0188] In addition, the electronic device 300 may further include a power supply component 302 and a communication component 304, wherein the power supply component 302 may be configured to perform power management of the electronic device 300, and the communication component 304 may be configured to implement communication of the electronic device 300, for example, wired or wireless communication. In addition, the electronic device 300 may further include an input / output (I / O) interface 305. The electronic device 300 may operate based on an operating system stored in the memory 303, such as Windows ServerTM, Mac OSXTM, UnixTM, LinuxTM, etc.
[0189] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned database table classification method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 303 including program instructions, and the above-mentioned program instructions can be executed by the processor 301 of the electronic device 300 to complete the above-mentioned database table classification method.
[0190] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.
[0191] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.
[0192] In addition, various different embodiments of the present disclosure can also be combined arbitrarily, as long as it does not violate the idea of the present disclosure, it should also be regarded as the content disclosed by the present disclosure. And in the present disclosure, relational terms such as first and second, etc. are only used to distinguish an entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements that are not clearly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.
Claims
1. A database table classification method, characterized in that: include: Obtain database table information of different industries and departments, and construct a training data set based on the database table information; Constructing a neural network model of a multi-layer composite structure; wherein the neural network model is used to determine the confidentiality level of the database table according to the information of the database table; Using the training data set to train the neural network model, to obtain a trained database table classification model; Acquire a database table to be classified, and input the database table to be classified into the database table classification model to obtain a confidentiality level of the database table to be classified; The neural network model is obtained by combining a first neural network structure, a second neural network structure, a third neural network structure and a fourth neural network structure; the first neural network structure is used to generate a semantic feature vector of a preset dimension based on the one-to-one correspondence between the department name, the library table name and the name of the information item in the database table, and add the semantic feature vectors corresponding to the information item name in the same database table to obtain a semantic feature aggregation vector, and map the semantic feature aggregation vector to obtain a first feature vector with unchanged dimension; the second neural network structure is used to combine the new features corresponding to the same database table to obtain a combined feature, and generate a second feature vector with the same dimension as the semantic feature vector based on the combined feature; the third neural network structure is used to classify the database table by industry based on the semantic feature vector corresponding to the department name of the database table; the fourth neural network structure is used to assign a preset vector weight value to the semantic feature vector corresponding to the department name and the library table name of the same database table, and the first feature vector and the second feature vector corresponding to the same database table, and add them to obtain a third feature vector, and perform coarse-grained classification and fine-grained classification on the same database table based on the third feature vector; wherein the classification types corresponding to the fine-grained classification are more than the classification types corresponding to the coarse-grained classification.
2. The database table classification method according to claim 1, characterized in that: The database table information includes the department names, table names, information item names in the tables, and the classification labels and industry labels corresponding to the different database tables.
3. The database table classification method according to claim 2, characterized in that: Constructing a training data set based on the database table information includes: Deduplication and removal of illegal information are performed on the department name, library table name and information item name in each database table; Randomly adding or removing information items that have no effect on the grading result from the information items contained in each of the database tables; Constructing new features for each information item contained in the database table; wherein the new features are feature variables associated with the information items in the table and having an impact on the classification results; The newly constructed features corresponding to each of the database tables are combined with the department name, library table name and information item name in each of the database tables after final processing, as well as the classification label and industry label corresponding to each of the database tables to form the training data set.
4. The database table classification method according to claim 1, characterized in that: Constructing the first neural network structure includes: Constructing an embedding layer; the embedding layer is used to map each character in the department name, library table name and information item name in the database table into a Chinese character vector; Constructing a two-layer bidirectional recurrent neural network; the recurrent neural network is used to convert each of the Chinese character vectors into a semantic feature vector of a preset dimension; Constructing a vector addition layer; the vector addition layer is used to add the semantic feature vectors corresponding to the information item names in the same database table to obtain a semantic feature aggregation vector; Constructing a first fully connected layer; the first fully connected layer is used to map the semantic feature aggregation vector to obtain a semantic feature mapping vector with unchanged dimension; Constructing a first dropout layer; the first dropout layer is used to map the semantic feature mapping vector to obtain a first feature vector with unchanged dimension.
5. The database table classification method according to claim 1, characterized in that: Constructing the second neural network structure, including: Constructing a second fully connected layer; the second fully connected layer is used to combine the new features constructed corresponding to the same database table to obtain combined features, and generate a new feature vector with the same dimension as the semantic feature vector based on the combined features; Constructing a batch normalization layer; the batch normalization layer is used to perform numerical normalization processing on the new feature vector to obtain a new feature numerical normalized vector; Constructing an activation function layer; the activation function layer is used to perform nonlinear processing on the new feature value normalized vector to obtain a new feature value normalized vector after nonlinear processing; Construct a second dropout layer; the second dropout layer is used to map the normalized vector of the new feature value after nonlinear processing to obtain a second feature vector with unchanged dimension.
6. The database table classification method according to claim 1, characterized in that: Constructing the third neural network structure, including: Constructing a third fully connected layer; the third fully connected layer is used to map the semantic feature vector corresponding to the department name in the database table to obtain a department feature mapping vector with unchanged dimension; Constructing a first classification layer; the first classification layer is used to classify the database table by industry based on the department feature mapping vector.
7. The database table classification method according to claim 1, characterized in that: Constructing the fourth neural network structure, including: Constructing a vector weight addition layer; the vector weight addition layer is used to assign preset vector weight values to the semantic feature vectors corresponding to the department name and the library table name of the same database table and the first feature vector and the second feature vector corresponding to the same database table, and add them to obtain a third feature vector; Constructing a fourth fully connected layer; the fourth fully connected layer is used to map the third feature vector to obtain a first mapping vector with unchanged dimension; Constructing a second classification layer; the second classification layer is used to perform coarse-grained classification on the same database table based on the first mapping vector; Constructing a fifth fully connected layer; the fifth fully connected layer is used to map the third feature vector to obtain a second mapping vector with unchanged dimension; A third classification layer is constructed; the third classification layer is used to perform fine-grained classification on the same database table based on the second mapping vector.
8. The database table classification method according to any one of claims 1 and 4-7, characterized in that: The neural network model is trained using the training data set to obtain a trained database table classification model, including: Inputting information of different database tables in the training data set into the neural network model in sequence, and obtaining industry classification results, coarse-grained classification results and fine-grained classification results corresponding to the different database tables; Substituting the industry classification result corresponding to the target database table and the industry label corresponding to the target database table into a preset loss function to calculate the loss, and obtaining a first loss; wherein the target database table is any database table in the training data set; Substituting the coarse-grained classification result corresponding to the target database table and the coarse-grained classification label corresponding to the target database table into a preset loss function to perform loss calculation to obtain a second loss; Substituting the fine-grained classification result corresponding to the target database table and the fine-grained classification label corresponding to the target database table into a preset loss function to perform loss calculation to obtain a third loss; Assigning a preset loss weight value to the first loss, the second loss, and the third loss and adding them together to obtain a total loss; With the optimization goal of reducing the total loss to 0, the parameters of the neural network model are optimized and adjusted using a preset neural network model optimization algorithm until all the training data sets are trained on the neural network model to obtain a trained database table classification model.
9. A database table grading device, characterized in that: include: A data construction unit, used to obtain database table information of different industries and departments, and to construct a training data set based on the database table information; A model building unit, used to build a neural network model of a multi-layer composite structure; wherein the neural network model is used to determine the confidentiality level of the database table according to the information of the database table; A model training unit, used to train the neural network model using the training data set to obtain a trained database table classification model; A database table grading unit, used for acquiring a database table to be graded, and inputting the database table to be graded into the database table grading model to obtain a confidentiality level of the database table to be graded; The neural network model is obtained by combining a first neural network structure, a second neural network structure, a third neural network structure and a fourth neural network structure; the first neural network structure is used to generate a semantic feature vector of a preset dimension based on the one-to-one correspondence between the department name, the library table name and the name of the information item in the database table, and add the semantic feature vectors corresponding to the information item name in the same database table to obtain a semantic feature aggregation vector, and map the semantic feature aggregation vector to obtain a first feature vector with unchanged dimension; the second neural network structure is used to combine the new features corresponding to the same database table to obtain a combined feature, and generate a second feature vector with the same dimension as the semantic feature vector based on the combined feature; the third neural network structure is used to classify the database table by industry based on the semantic feature vector corresponding to the department name of the database table; the fourth neural network structure is used to assign a preset vector weight value to the semantic feature vector corresponding to the department name and the library table name of the same database table, and the first feature vector and the second feature vector corresponding to the same database table, and add them to obtain a third feature vector, and perform coarse-grained classification and fine-grained classification on the same database table based on the third feature vector; wherein the classification types corresponding to the fine-grained classification are more than the classification types corresponding to the coarse-grained classification.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the database table grading method according to any one of claims 1 to 8 are implemented.
11. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of the database table grading method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Report sensitivity grading method and device
CN110008470A
Field label obtaining method and device, storage medium and electronic device
CN110442568A