Data identification method, network device and storage medium

Abstract

The embodiment of the invention provides a data identification method, network equipment and a storage medium, and relates to the technical field of communication. The method comprises the following steps: acquiring target field information corresponding to target data, and identifying the target field information based on a first classification model to obtain a first category identification result corresponding to the target field information; if it is determined that the first category recognition result is recognition failure, feature extraction is conducted on the target field information, and field feature information is obtained; based on a second classification model, identifying the target field information according to the field feature information to obtain a second category identification result corresponding to the target field information; and if it is determined that the second category identification result is successful identification, determining security level information corresponding to the target field information according to the second category identification result, and generating a data identification result according to the target field information and the corresponding category and security level information thereof. According to the method, the category of the sensitive data can be accurately determined, and the data recognition efficiency and precision are improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a data identification method, a network device and a storage medium. Background Art

[0002] With the continuous development of information technology, massive amounts of data are generated every day. Data tables and data fields are named by business personnel according to their own habits, and no unified standards have been formed. Therefore, the existing methods for identifying data sensitive information categories usually involve a lot of manual operations: in addition to visually observing column names, field explanations and corresponding data content to determine the category to which the field belongs, experienced data managers also need to manually enter regular expressions that match data content or table names and column names in the database to create data classification rules, thereby providing batch processing capabilities for data that meets the rules and has a specified confidentiality level.

[0003] However, existing methods for identifying data sensitive information categories have disadvantages such as difficulty in classifying sensitive information and difficulty in obtaining category labels. They also require a large number of data processing personnel with business experience to build a huge sensitive vocabulary library, and the data processing efficiency is low and the accuracy cannot be guaranteed. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide a data processing method, a network device and a storage medium, which aim to identify sensitive data from different dimensions through different models, ensure that the category of sensitive data can be accurately determined, and improve data identification efficiency and accuracy.

[0005] In a first aspect, an embodiment of the present application provides a data identification method, the method comprising:

[0006] Acquire target field information corresponding to the target data, and identify the target field information based on a first classification model to obtain a first category recognition result corresponding to the target field information;

[0007] If it is determined that the first category recognition result is unsuccessful, extracting features from the target field information to obtain field feature information;

[0008] Based on the second classification model, the target field information is identified according to the field feature information to obtain a second category identification result corresponding to the target field information;

[0009] If it is determined that the second category identification result is successful, the confidentiality level information corresponding to the target field information is determined according to the second category identification result, and a data identification result is generated according to the target field information and its corresponding category and confidentiality level information.

[0010] In a second aspect, an embodiment of the present application further provides a network device, the network device comprising:

[0011] A processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of the data identification method as described above are realized.

[0012] In a third aspect, an embodiment of the present application further provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the data identification method as described above.

[0013] The embodiment of the present application provides a data identification method, a network device and a storage medium, which obtains the target field information corresponding to the target data, and identifies the target field information based on a first classification model to obtain a first category identification result corresponding to the target field information; if the first category identification result is determined to be an unsuccessful identification, then the target field information is feature extracted to obtain field feature information; based on a second classification model, the target field information is identified according to the field feature information to obtain a second category identification result corresponding to the target field information; if the second category identification result is determined to be a successful identification, the confidentiality level information corresponding to the target field information is determined according to the second category identification result, and a data identification result is generated according to the target field information and its corresponding category and confidentiality level information. In this way, sensitive data can be identified from different dimensions through different models to ensure that the category of sensitive data can be accurately determined, improve data identification efficiency and accuracy, thereby protecting sensitive data and avoiding leakage of important information. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 A flowchart of a data identification method provided in an embodiment of the present application;

[0016] Figure 2 A flowchart of how to obtain a first category recognition result corresponding to target field information provided in an embodiment of the present application;

[0017] Figure 3 A schematic diagram of the structure of a second classification model provided in an embodiment of the present application;

[0018] Figure 4 A flowchart of the overall process of a data identification method provided in an embodiment of the present application;

[0019] Figure 5 A schematic block diagram of the structure of a data identification device provided in an embodiment of the present application;

[0020] Figure 6 A schematic block diagram of the structure of a network device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0023] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0024] For tabular data, such as massive data from multiple industries such as banking, manufacturing, government and enterprises, and telecommunications, data governance demanders need to analyze information such as data table names, field names, and field contents, and use methods such as manual recognition, regular matching, or AI model automatic recognition to distinguish data categories and identify the corresponding sensitivity level of the data.

[0025] In the related technology, the segmented words extracted from the source data can be matched one by one with the same sensitive words and similar sensitive words in the preset database to determine that the segmented words are sensitive data. It is also possible to establish a regular expression rule library for sensitive data identification, and scan the field sets to be identified separately until all the fields to be identified are scanned. However, the above two sensitive data identification methods require a large number of data processing personnel with business experience to build a huge sensitive word library, and the data processing efficiency is low, and the accuracy cannot be guaranteed.

[0026] At present, some technologies have proposed to centrally process structured data through machine learning algorithms or deep learning algorithms, which can effectively improve data processing efficiency, reduce personnel business and technical requirements, and significantly reduce manual workload. For example, sensitive data features can be input into a neural network classification model, and sensitive data identification models containing decision trees and support vector machine models can be used to classify sensitive information. However, the above two methods rely on pre-trained classification models. Once new and more data are generated, the model cannot be iterated and evolved quickly, and can only rely on the old model. The accuracy cannot be guaranteed, and the governance task is unsustainable.

[0027] The embodiment of the present application provides a data identification method, a network device and a storage medium. The method can be applied to a network device, wherein the network device can be a server, a base station, a switch, a router, an IPRAN product and an ITN product, etc., but is not limited thereto. Thus, sensitive data can be identified from different dimensions through different models, ensuring that the category of sensitive data can be accurately determined, improving data identification efficiency and accuracy, thereby protecting sensitive data and avoiding leakage of important information.

[0028] Please refer to Figure 1 , Figure 1 A flow chart of a data identification method provided in an embodiment of the present application. The data identification method can identify sensitive data from different dimensions through different models, ensure that the category of sensitive data can be accurately determined, improve data identification efficiency and accuracy, thereby protecting sensitive data and avoiding leakage of important information.

[0029] like Figure 1 As shown, the data identification method may include S101 to S104.

[0030] S101. Obtain target field information corresponding to target data, and identify the target field information based on a first classification model to obtain a first category recognition result corresponding to the target field information.

[0031] Among them, the target data can be structured data, which can come from a database or offline data. The target field information can include attribute name information and attribute value information. The attribute name information can be a table name, column name, etc. The attribute value information can be the specific content of the attribute, which can be information of data types such as float, string, timestamp and bigint. The first classification model is an offline recognition model, which can be a rule model, which is mainly used to quickly solve the classification and recognition of data with relatively fixed semantic features, including the matching of attribute name information and attribute value information. The first category recognition result is the result of identifying the target field information through the first classification model.

[0032] In some embodiments, target data is acquired, and field information is extracted from the target data; the format of the field information is standardized to obtain the target field information, thereby accurately extracting the target field information from the target data.

[0033] Among them, the target data is sparse and heterogeneous. Sparse means that there may be a large number of zero values ​​or null values ​​in the data. Heterogeneous means that the data is often composed of a variety of different data types, such as text data, numerical data, category data, image data, etc., and the above data can all be target data. The field information can be attribute name information and attribute value information in non-uniform formats. The target field information is attribute name information and attribute value information in a unified format, and the target field information can be directly sent to the machine learning or deep learning model for recognition and learning.

[0034] Specifically, before obtaining the target data, you need to set the database read permission, or the user can set the offline data reading path by himself, and parse the obtained data into a comma-delimited csv file for subsequent processing.

[0035] Specifically, to obtain the target data, attribute name information and attribute value information can be extracted from the target data according to the data type; then the format of the field information is standardized, such as processing missing values, invalid values, duplicate values, stop words, special characters, etc., to obtain the target field information.

[0036] Exemplarily, each column of the target data has 1 to N possible values, where N depends on the number of rows. Even if the values ​​of each row are different, the semantics of each column are the same. The target data may have tens of thousands of rows or even more, so the collected target data needs to be intercepted first. Missing values, invalid values, duplicate values, stop words, special characters, etc. need to be removed separately, and missing values ​​can also be filled with neighbor values. For English texts with uppercase and lowercase, they need to be processed into a unified format. For the convenience of subsequent processing, it is necessary to segment long Chinese and English sentences according to a certain strategy, such as Jieba word segmentation, and perform word form restoration, stem extraction, and part-of-speech tagging on the word vectors after word segmentation. In addition to sensitive data such as personal information, the target data may also include some automatically added information when the table is created, such as the time of table creation, the person who created the table, etc. This type of information can be classified into the same category and level and no subsequent processing is performed.

[0037] Illustratively, Table 1 is a comparison result before and after the format of field information is standardized.

[0038]

[0039] Table 1

[0040] It can be seen from Table 1 that before standardization, the formats of attribute name information and attribute value information are not unified, which is not conducive to the classification model to identify. After standardization, the formats of attribute name information and attribute value information can be unified into a fixed format, making it easier for the classification model to identify categories.

[0041] In some embodiments, the information format of the attribute name information and the attribute value information is determined; if the information format of the attribute name information or the attribute value information is a first preset information format, the attribute name information or the attribute value information is matched based on a preset regular expression to obtain a category recognition result corresponding to the attribute name information or the attribute value information; if the information format of the attribute name information or the attribute value information is a second preset information format, the attribute name information or the attribute value information is subjected to named entity recognition and matching based on a preset corpus to obtain a category recognition result corresponding to the attribute name information or the attribute value information; based on the category recognition result corresponding to the attribute name information and the attribute value information, the first category recognition result corresponding to the target field information is determined. In this way, the first category recognition result can be accurately determined according to the information format.

[0042] Among them, the information format can be a unique format for attribute name information or attribute value information. The first preset information format can be a number or character string with fixed digits in a mobile phone number, such as containing keywords such as "-", "86", etc., or gender, such as keywords involving "male", "female", "female", "male", "woman", "man", etc. The second preset information format is an information format that can be used to determine the category to which the field belongs by exhaustive enumeration, such as a name, place name, etc. The preset regular expression can be used for fuzzy or precise matching of attribute name information or attribute value information. The preset corpus can be saved in txt file format in the offline stage, uniformly stored in a fixed path, and a sample library can be constructed to facilitate addition and modification and speed up recognition.

[0043] Specifically, first determine the information format of the attribute name information and the attribute value information; if the information format of the attribute name information and the attribute value information are both the first preset information format, match the attribute name information and the attribute value information based on the preset regular expression to obtain the category identification result corresponding to the attribute name information and the attribute value information.

[0044] If the information format of the attribute name information is the first preset information format, and the information format of the attribute value information is the second preset information format, the attribute name information is matched based on the preset regular expression to obtain the category recognition result corresponding to the attribute name information, and the attribute value information is named entity recognized and matched based on the preset corpus to obtain the category recognition result corresponding to the attribute value information.

[0045] If the information format of the attribute name information is the second preset information format, and the information format of the attribute value information is the first preset information format, the attribute value information is matched based on the preset regular expression to obtain the category recognition result corresponding to the attribute value information, and the attribute name information is subjected to named entity recognition and matching based on the preset corpus to obtain the category recognition result corresponding to the attribute name information.

[0046] If the information formats of the attribute name information and the attribute value information are both the second preset information format, named entity recognition and matching are performed on the attribute name information and the attribute value information based on the preset corpus to obtain category recognition results corresponding to the attribute name information and the attribute value information.

[0047] Exemplarily, if the information format of the attribute value information is telephone, it can be determined that the information format of the attribute value information is the first preset information format, and the attribute value information is fuzzy or precisely matched based on the preset regular expression. If the preset regular expression can be matched, it can be determined that the category recognition result of the attribute value information is successful and the category value corresponding to the attribute value information is returned; if the preset regular expression cannot be matched, the returned category value is an empty string.

[0048] Exemplarily, if the information format of the attribute name information is a place name, it can be determined that the information format of the attribute name information is the second preset information format, and named entity recognition and matching are performed on the attribute name information based on the preset corpus. If it can be matched in the preset corpus, it can be determined that the category recognition result of the attribute name information is a successful recognition and the category value corresponding to the attribute name information is returned; if it cannot be matched in the preset corpus, the returned category value is an empty string.

[0049] like Figure 2 As shown, Figure 2 A flowchart of how to obtain a first category recognition result corresponding to target field information is provided in an embodiment of the present application.

[0050] S201. Determine the information format of the attribute name information and the attribute value information. If the information format of the attribute name information or the attribute value information is the first preset information format, go to S202; if the information format of the attribute name information or the attribute value information is the second preset information format, go to S203.

[0051] S202: Match the attribute name information or the attribute value information based on a preset regular expression to obtain a category recognition result corresponding to the attribute name information or the attribute value information.

[0052] S203: Perform named entity recognition and matching on the attribute name information or the attribute value information based on a preset corpus to obtain a category recognition result corresponding to the attribute name information or the attribute value information.

[0053] S204: Determine a first category recognition result corresponding to the target field information according to the category recognition result corresponding to the attribute name information and the attribute value information.

[0054] In some embodiments, if the category recognition result corresponding to the attribute name information and / or the category recognition result corresponding to the attribute value information is a successful recognition, the first category recognition result corresponding to the target field information is determined to be a successful recognition, and the category recognition result corresponding to the successful recognition is used as the category corresponding to the target field information; if the category recognition result corresponding to the attribute name information and the category recognition result corresponding to the attribute value information are both unsuccessful, the first category recognition result corresponding to the target field information is determined to be an unsuccessful recognition. In this way, the first category recognition result corresponding to the target field information can be accurately determined.

[0055] Among them, the category corresponding to the target field information may include a sensitive information category or a non-sensitive information category. If the target field is personal information such as an ID card, it can be considered a sensitive information category.

[0056] Exemplarily, the attribute name information or attribute value information can be matched based on a preset regular expression, or named entity recognition and matching can be performed on the attribute name information or attribute value information based on a preset corpus; thereby obtaining category recognition results corresponding to the attribute name information and attribute value information.

[0057] If the category identification result corresponding to the attribute name information and / or the category identification result corresponding to the attribute value information is a successful identification, then the first category identification result corresponding to the target field information is determined to be a successful identification, and the category identification result corresponding to the successful identification is parsed to obtain the corresponding category value, and the category corresponding to the category value is determined to determine whether it is a sensitive information category, thereby determining the category corresponding to the target field information.

[0058] If the category recognition result corresponding to the attribute name information and the category recognition result corresponding to the attribute value information are both unsuccessful, that is, the returned category values ​​are both empty strings, then it is determined that the first category recognition result corresponding to the target field information is unsuccessful.

[0059] S102: If it is determined that the first category recognition result is unsuccessful, feature extraction is performed on the target field information to obtain field feature information.

[0060] Among them, the field feature information can be used to represent the characteristics of the target field information, for example, it can include word frequency vector features, statistical features, word vector features and entropy value features.

[0061] In some embodiments, feature extraction is performed on the attribute name information to obtain word frequency vector features, and the information type of the attribute value information is determined; if the information type of the attribute value information is a numerical type, feature extraction is performed on the attribute value information to obtain statistical features; if the information type of the attribute value information is a non-numerical type, feature extraction is performed on the attribute value information to obtain word vector features and entropy features. In this way, the field feature information corresponding to the attribute name information and the attribute value information can be accurately extracted.

[0062] Among them, the word frequency vector feature can be used to indicate the number of times a word appears in a text. If a word appears more frequently in one text but less frequently in another text, the word is more important to the text.

[0063] Exemplarily, the TF-IDF word frequency vector feature of the attribute name information is first calculated, so as to characterize the number of occurrences of the attribute name information in the text. Then determine whether the information type of the attribute value information is a numerical type or a non-numerical type. If the information type of the attribute value information is a numerical type, feature extraction is performed on the attribute value information to obtain statistical features such as mean, variance, maximum, minimum, etc.; if the information type of the attribute value information is a non-numerical type, feature extraction is performed on the attribute value information to obtain word vector features such as word2vec word vectors and entropy features such as the entropy value of the attribute value information.

[0064] S103: Based on the second classification model, identify the target field information according to the field feature information to obtain a second category identification result corresponding to the target field information.

[0065] The second classification model may be a trained benchmark model such as a textCNN model. The second classification model may include an embedding layer, a convolution layer, a pooling layer, and a fully connected layer. The second category recognition result is the result of identifying the target field information through the second classification model.

[0066] Specifically, the prerequisite for the second classification model recognition is that the models of some categories have been trained, and when the data has no confirmed category output after being recognized by the rule model, the second classification model needs to be used. The structure of the second classification model is as follows Figure 3 As shown in the figure, the model is simple and effective, with few parameters and fast convergence speed.

[0067] In some embodiments, based on the embedding layer, the field feature information is mapped to obtain a word vector matrix of a preset dimension; based on the convolution layer, the word vector matrix of the preset dimension is feature extracted to obtain word vector matrix features; based on the pooling layer, the word vector matrix features are pooled to obtain target word vector matrix features; based on the fully connected layer, recognition is performed according to the target word vector matrix features to obtain the second category recognition result corresponding to the target field information. In this way, the second category recognition result corresponding to the target field information can be accurately determined.

[0068] Among them, the embedding layer is used to establish the word vector relationship in the field feature information to construct the word vector matrix. The preset dimension is used to indicate the dimension of the word vector matrix, which can be any dimension and is not specifically limited here. The convolution layer is used to extract the feature information of the word vector matrix. The pooling layer is used to sample the word vector matrix features, reduce the dimension, remove redundant information, compress the features, simplify the network complexity, reduce the amount of calculation, and reduce memory consumption. The target word vector matrix features are the word vector matrix features after being filtered by the pooling layer. The fully connected layer is used for classification to determine the second category recognition result corresponding to the target field information.

[0069] Specifically, word vectors are first extracted from the field feature information, and the word vectors are padded to a fixed length. An embedding layer is used to map each word into an N-dimensional word vector, thereby generating a word vector matrix of preset dimensions. Then, three convolution kernels of different sizes are used to extract features from the word vector matrix respectively. Next, the extracted word vector matrix features are input into a one-dimensional maximum pooling layer. Finally, the downsampled target word vector matrix features are concatenated and sent to the fully connected layer to obtain the second category recognition result.

[0070] In order to ensure the accuracy of the second classification model, a word vector dataset for training, verification and testing is constructed. The data distribution ratio can be 7:2:1, and it is randomly shuffled. The second classification model is input for training and saved.

[0071] It should be noted that if the second category identification result corresponding to the target field information is a successful identification, the confidentiality level information corresponding to the target field information is determined based on the second category identification result, and the data identification result is generated based on the target field information and its corresponding category and confidentiality level information; if the second category identification result corresponding to the target field information is a failed identification, the corresponding return value is an empty string.

[0072] S104: If it is determined that the second category recognition result is successful, determine the confidentiality level information corresponding to the target field information according to the second category recognition result, and generate a data recognition result according to the target field information and its corresponding category and confidentiality level information.

[0073] The confidentiality level information may be determined according to the data category. Generally speaking, sensitive data corresponds to a higher confidentiality level. The data identification result may include target field information and its corresponding category and confidentiality level information, etc., for displaying the above information to the user.

[0074] Exemplarily, if the category corresponding to the target field information is sensitive information, then the confidentiality level information corresponding to the target field information is determined to be set to the first confidentiality level; if the category corresponding to the target field information is non-sensitive information, then the confidentiality level information corresponding to the target field information is determined to be set to the second confidentiality level, thereby determining the data governance policy corresponding to the confidentiality level information; wherein the confidentiality level of the first confidentiality level is greater than the confidentiality level of the second confidentiality level.

[0075] In some embodiments, based on the second classification model, the target field information is identified according to the field feature information, and after obtaining the second category identification result corresponding to the target field information, if the second category identification result is determined to be unsuccessful, the target field information is clustered and the category is predicted based on the third classification model according to the field feature information to obtain the third category identification result corresponding to the target field information; if the third category identification result is determined to be successful, the confidentiality level information corresponding to the target field information is determined according to the third category identification result, and the data identification result is generated according to the target field information and its corresponding category and confidentiality level information. In this way, the data that is difficult to identify can be classified into different clusters through the third classification model, and the text describing the category with the highest probability of the current cluster can be output, so as to facilitate the data management personnel to correct errors.

[0076] The third classification model can be used to classify the difficult-to-recognize data into different clusters and output a text description of the category with the highest probability of belonging to the current cluster, such as a text clustering model. The third category recognition result can be the result of recognizing the target field information through the third classification model.

[0077] Specifically, a prerequisite for the third classification model to be identified is that if there is no confirmed category output after the data has been identified by the first classification model and the second classification model, the third classification model needs to be used for identification.

[0078] Specifically, the target field information is clustered and the category is predicted according to the field feature information to obtain a third category recognition result corresponding to the target field information; if the third category recognition result is determined to be a successful recognition, the confidentiality level information corresponding to the target field information is determined according to the third category recognition result, and a data recognition result is generated according to the target field information and its corresponding category and confidentiality level information; if the third category recognition result is determined to be an unsuccessful recognition, the target field information needs to be re-recognized, or the target field information needs to be marked for manual recognition.

[0079] Exemplarily, the word vector features, entropy features and word frequency vector features in the field feature information are used as inputs to the third classification model. After processing, if the field name of a cluster points to a specific category, the category of this cluster is assigned to this specific category result. If it does not point to a specific category, the category of this cluster is uniformly set to the "other" category. In this way, after the difficult-to-identify data is classified into different clusters, the output text describes the category with the highest probability of belonging to the current cluster.

[0080] In some embodiments, if the first category recognition result or the second category recognition result is a successful recognition, the corresponding target data is obtained, and the target data is cleaned to obtain standard data; the second classification model is trained according to the standard data to update the model parameters of the second classification model to obtain the target second classification model. In this way, the standard data can be screened to retrain the second classification model to quickly iterate and update the second classification model.

[0081] The standard data is data for determining the category of the corresponding target field information, and is used to train the second classification model, such as the cleaned target data. The model parameters may include parameters such as model error. The target second classification model is the second classification model after iterative update.

[0082] Specifically, if the first category recognition result or the second category recognition result is a successful recognition, it can be determined that the first classification model or the second classification model can determine the category of the target field information. Therefore, the target data corresponding to the first category recognition result or the second category recognition result can be obtained, and the target data can be cleaned to obtain standard data; then the second classification model is trained according to the standard data to iteratively update the model parameters of the second classification model to obtain the target second classification model. In this way, the data that has been recognized and does not need error correction and the data that has been cleaned after error correction can be used as training data to realize online training and updating of the second classification model, thereby reducing the workload of expert annotation, improving recognition efficiency and accuracy, and adapting to new data forms and content, quickly iterating the model, and gradually realizing the governance of all data.

[0083] It should be noted that if the third category recognition result is successful, the target data corresponding to the third classification model can also be obtained, and the target data can be cleaned to obtain standard data; then the second classification model can be trained according to the standard data to iteratively update the model parameters of the second classification model to obtain the target second classification model.

[0084] After the above steps, each classification model will output the category to which the current field belongs, recorded as the "label" column result. In addition, it will also output the category to which the field may belong with a higher probability, and use it as the "probability" column result. Set an "other" class to represent that a field does not belong to all the existing categories of all current classifications. If the "label" value of the category to which the field belongs is not the "other" class, the "probability" column of the field will also output the same result as the "label" column; if the "label" value of the field is the "other" class, then calculate the mode of all non-"other" element identification categories under the field, fill it into the "probability" column of the field, and match it with the currently set data confidentiality policy to obtain the confidentiality information of the field in the "security" column. In addition to the "label" column and the "probability" column, you also need to randomly select two desensitized sample data and output them together, such as

[0085] As shown in Table 2, Table 2 is used to represent the field names in the data recognition results of the field to be recognized (target field information).

[0086]

[0087]

[0088] Table 2

[0089] In some embodiments, Figure 4 As shown, the data identification method proposed in an embodiment of the present application is introduced from the overall process of data identification, including S301-S308.

[0090] S301. Obtain target field information corresponding to target data, and identify the target field information based on a first classification model to obtain a first category recognition result corresponding to the target field information. If the first category recognition result is a successful recognition, go to S302; if the first category recognition result is a failed recognition, go to S303.

[0091] S302. Determine the confidentiality level information corresponding to the target field information according to the first category recognition result, and generate a data recognition result according to the target field information and its corresponding category and confidentiality level information.

[0092] S303: Extract features of the target field information to obtain field feature information.

[0093] S304. Based on the second classification model, the target field information is identified according to the field feature information to obtain a second category identification result corresponding to the target field information. If the second category identification result is a successful identification, go to S305; if the second category identification result is a failed identification, go to S306.

[0094] S305. Determine the confidentiality level information corresponding to the target field information according to the second category recognition result, and generate a data recognition result according to the target field information and its corresponding category and confidentiality level information.

[0095] S306. Based on the third classification model, cluster the target field information and predict the category according to the field feature information to obtain the third category recognition result corresponding to the target field information. If the third category recognition result is a successful recognition, go to S307; if the third category recognition result is a failed recognition, go to S308.

[0096] S307. Determine the confidentiality level information corresponding to the target field information according to the third category recognition result, and generate a data recognition result according to the target field information and its corresponding category and confidentiality level information.

[0097] S308: Re-identify the target field information, or mark the target field information for manual identification.

[0098] The embodiment of the present application provides a data identification method, which obtains the target field information corresponding to the target data, and identifies the target field information based on a first classification model to obtain a first category identification result corresponding to the target field information; if the first category identification result is determined to be an unsuccessful identification, then feature extraction is performed on the target field information to obtain field feature information; based on a second classification model, the target field information is identified according to the field feature information to obtain a second category identification result corresponding to the target field information; if the second category identification result is determined to be a successful identification, the confidentiality level information corresponding to the target field information is determined according to the second category identification result, and a data identification result is generated according to the target field information and its corresponding category and confidentiality level information. In this way, sensitive data can be identified from different dimensions through different models to ensure that the category of sensitive data can be accurately determined, improve data identification efficiency and accuracy, thereby protecting sensitive data and avoiding leakage of important information.

[0099] See also Figure 5 , Figure 5 It is a schematic block diagram of a data identification device provided in an embodiment of the present application. The data identification device can be configured in a server to execute the aforementioned data identification method.

[0100] like Figure 5As shown, the data identification device 400 includes: a first classification module 401, a feature extraction module 402, a second classification module 403 and a confidentiality level determination module 404.

[0101] The first classification module 401 is used to obtain target field information corresponding to the target data, and identify the target field information based on a first classification model to obtain a first category recognition result corresponding to the target field information;

[0102] A feature extraction module 402 is used to extract features from the target field information to obtain field feature information if it is determined that the first category recognition result is unsuccessful recognition;

[0103] A second classification module 403 is used to identify the target field information according to the field feature information based on a second classification model to obtain a second category recognition result corresponding to the target field information;

[0104] The confidentiality level determination module 404 is used to determine the confidentiality level information corresponding to the target field information according to the second category identification result if it is determined that the second category identification result is successful, and generate a data identification result according to the target field information and its corresponding category and confidentiality level information.

[0105] The first classification module 401 is also used to determine the information format of the attribute name information and the attribute value information; if the information format of the attribute name information or the attribute value information is the first preset information format, the attribute name information or the attribute value information is matched based on a preset regular expression to obtain a category recognition result corresponding to the attribute name information or the attribute value information; if the information format of the attribute name information or the attribute value information is the second preset information format, the attribute name information or the attribute value information is named entity recognized and matched based on a preset corpus to obtain a category recognition result corresponding to the attribute name information or the attribute value information; according to the category recognition results corresponding to the attribute name information and the attribute value information, the first category recognition result corresponding to the target field information is determined.

[0106] The first classification module 401 is also used to determine that the first category recognition result corresponding to the target field information is a successful recognition if the category recognition result corresponding to the attribute name information and / or the category recognition result corresponding to the attribute value information is a successful recognition, and use the category recognition result corresponding to the successful recognition as the category corresponding to the target field information; if the category recognition result corresponding to the attribute name information and the category recognition result corresponding to the attribute value information are both unsuccessful recognition, then determine that the first category recognition result corresponding to the target field information is an unsuccessful recognition.

[0107] The feature extraction module 402 is also used to perform feature extraction on the attribute name information to obtain word frequency vector features and determine the information type of the attribute value information; if the information type of the attribute value information is a numerical type, then feature extraction is performed on the attribute value information to obtain statistical features; if the information type of the attribute value information is a non-numerical type, then feature extraction is performed on the attribute value information to obtain word vector features and entropy features.

[0108] The second classification module 403 is also used to map the field feature information based on the embedding layer to obtain a word vector matrix of a preset dimension; based on the convolution layer, perform feature extraction on the word vector matrix of the preset dimension to obtain word vector matrix features; based on the pooling layer, perform pooling on the word vector matrix features to obtain target word vector matrix features; based on the fully connected layer, perform recognition according to the target word vector matrix features to obtain a second category recognition result corresponding to the target field information.

[0109] The third classification module 405 is used to cluster and predict the category of the target field information based on the field feature information based on the third classification model if it is determined that the second category recognition result is an unsuccessful recognition, and obtain a third category recognition result corresponding to the target field information; if it is determined that the third category recognition result is a successful recognition, determine the confidentiality level information corresponding to the target field information based on the third category recognition result, and generate a data recognition result based on the target field information and its corresponding category and confidentiality level information.

[0110] The model update module 406 is used to obtain the corresponding target data if the first category recognition result or the second category recognition result is a successful recognition, and clean the target data to obtain standard data; train the second classification model according to the standard data to update the model parameters of the second classification model to obtain the target second classification model.

[0111] The first classification module 401 is also used to obtain target data and extract field information from the target data; standardize the format of the field information to obtain target field information.

[0112] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0113] The method and apparatus of the present application can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer terminal devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices.

[0114] See also Figure 6 , Figure 6 A schematic block diagram of the structure of a network device provided in an embodiment of the present application.

[0115] like Figure 6 As shown, the network device 500 may include a processor 501 and a memory 502 , and the processor 501 and the memory 502 are connected via a bus 503 , such as an I2C (Inter-integrated Circuit) bus.

[0116] In an exemplary embodiment, the processor 501 can be used to provide computing and control capabilities to support the operation of the entire terminal device. The processor 501 can be a central processing unit (CPU), and the processor 501 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0117] Specifically, the memory 502 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0118] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a partial structure related to the embodiment of the present application, and does not constitute a limitation on the terminal device to which the embodiment of the present application is applied. The specific server may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0119] The processor 501 is used to run a computer program stored in the memory, and implement any one of the data identification methods provided in the embodiments of the present application when executing the computer program.

[0120] In one embodiment, the processor 501 is used to run a computer program stored in a memory, and implement the following steps when executing the computer program: obtaining target field information corresponding to the target data, and identifying the target field information based on a first classification model to obtain a first category recognition result corresponding to the target field information; if it is determined that the first category recognition result is an unsuccessful recognition, performing feature extraction on the target field information to obtain field feature information; based on a second classification model, identifying the target field information according to the field feature information to obtain a second category recognition result corresponding to the target field information; if it is determined that the second category recognition result is a successful recognition, determining the confidentiality level information corresponding to the target field information according to the second category recognition result, and generating a data recognition result based on the target field information and its corresponding category and confidentiality level information.

[0121] In one embodiment, when the processor 501 implements the identification of the target field information based on the first classification model to obtain the first category identification result corresponding to the target field information, it is used to implement: determining the information format of the attribute name information and the attribute value information; if the information format of the attribute name information or the attribute value information is a first preset information format, matching the attribute name information or the attribute value information based on a preset regular expression to obtain the category identification result corresponding to the attribute name information or the attribute value information; if the information format of the attribute name information or the attribute value information is a second preset information format, performing named entity recognition and matching on the attribute name information or the attribute value information based on a preset corpus to obtain the category identification result corresponding to the attribute name information or the attribute value information; and determining the first category identification result corresponding to the target field information based on the category identification results corresponding to the attribute name information and the attribute value information.

[0122] In one embodiment, when the processor 501 implements the determination of the first category identification result corresponding to the target field information based on the category identification result corresponding to the attribute name information and the attribute value information, it is used to implement: if the category identification result corresponding to the attribute name information and / or the category identification result corresponding to the attribute value information is a successful identification, then the first category identification result corresponding to the target field information is determined to be a successful identification, and the category identification result corresponding to the successful identification is used as the category corresponding to the target field information; if the category identification result corresponding to the attribute name information and the category identification result corresponding to the attribute value information are both unsuccessful identification, then the first category identification result corresponding to the target field information is determined to be an unsuccessful identification.

[0123] In one embodiment, when the processor 501 implements the feature extraction of the target field information to obtain the field feature information, it is used to implement: feature extraction of the attribute name information to obtain word frequency vector features, and determine the information type of the attribute value information; if the information type of the attribute value information is a numerical type, feature extraction of the attribute value information to obtain statistical features; if the information type of the attribute value information is a non-numerical type, feature extraction of the attribute value information to obtain word vector features and entropy features.

[0124] In one embodiment, when the processor 501 implements the second classification model, identifies the target field information according to the field feature information, and obtains the second category recognition result corresponding to the target field information, it is used to implement: based on the embedding layer, mapping the field feature information to obtain a word vector matrix of a preset dimension; based on the convolution layer, performing feature extraction on the word vector matrix of the preset dimension to obtain word vector matrix features; based on the pooling layer, pooling the word vector matrix features to obtain target word vector matrix features; based on the fully connected layer, identifying according to the target word vector matrix features to obtain the second category recognition result corresponding to the target field information.

[0125] In one embodiment, after the processor 501 implements the second classification model-based identification of the target field information according to the field characteristic information to obtain the second category identification result corresponding to the target field information, it is used to implement: if it is determined that the second category identification result is unsuccessful, based on the third classification model, clustering and category prediction are performed on the target field information according to the field characteristic information to obtain a third category identification result corresponding to the target field information; if it is determined that the third category identification result is successful, determining the confidentiality level information corresponding to the target field information according to the third category identification result, and generating a data identification result based on the target field information and its corresponding category and confidentiality level information.

[0126] In one embodiment, the processor 501 is also used to implement: if the first category recognition result or the second category recognition result is a successful recognition, then the corresponding target data is obtained, and the target data is cleaned to obtain standard data; the second classification model is trained according to the standard data to update the model parameters of the second classification model to obtain the target second classification model.

[0127] In one embodiment, when implementing the acquisition of target field information corresponding to target data, the processor 501 is used to: acquire the target data and extract field information from the target data; and standardize the format of the field information to obtain the target field information.

[0128] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working process of the network device described above can refer to the corresponding process in the aforementioned terminal capability reporting method embodiment, and will not be repeated here.

[0129] The embodiment of the present application also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any data identification method provided in the description of the embodiment of the present application.

[0130] The storage medium may be an internal storage unit of the network device described in the foregoing embodiment, such as a hard disk or memory of the network device. The storage medium may also be an external storage device of the network device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc., provided on the network device.

[0131] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0132] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0133] The serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A data identification method, characterized in that: The method comprises: Acquire target field information corresponding to the target data, and identify the target field information based on a first classification model to obtain a first category recognition result corresponding to the target field information; If it is determined that the first category recognition result is unsuccessful, extracting features from the target field information to obtain field feature information; Based on the second classification model, the target field information is identified according to the field feature information to obtain a second category identification result corresponding to the target field information; If it is determined that the second category identification result is successful, the confidentiality level information corresponding to the target field information is determined according to the second category identification result, and a data identification result is generated according to the target field information and its corresponding category and confidentiality level information.

2. The method according to claim 1, characterized in that: The target field information includes attribute name information and attribute value information, and the identifying the target field information based on the first classification model to obtain a first category identification result corresponding to the target field information includes: Determining the information format of the attribute name information and the attribute value information; If the information format of the attribute name information or the attribute value information is a first preset information format, matching the attribute name information or the attribute value information based on a preset regular expression to obtain a category identification result corresponding to the attribute name information or the attribute value information; If the information format of the attribute name information or the attribute value information is the second preset information format, performing named entity recognition and matching on the attribute name information or the attribute value information based on a preset corpus to obtain a category recognition result corresponding to the attribute name information or the attribute value information; According to the category identification results corresponding to the attribute name information and the attribute value information, a first category identification result corresponding to the target field information is determined.

3. The method according to claim 2, characterized in that The determining, according to the category identification results corresponding to the attribute name information and the attribute value information, a first category identification result corresponding to the target field information includes: If the category recognition result corresponding to the attribute name information and / or the category recognition result corresponding to the attribute value information is a successful recognition, then determining that the first category recognition result corresponding to the target field information is a successful recognition, and using the category recognition result corresponding to the successful recognition as the category corresponding to the target field information; If both the category recognition result corresponding to the attribute name information and the category recognition result corresponding to the attribute value information are unsuccessful recognition, it is determined that the first category recognition result corresponding to the target field information is unsuccessful recognition.

4. The method according to claim 1, characterized in that The target field information includes attribute name information and attribute value information. The feature extraction of the target field information to obtain field feature information includes: Performing feature extraction on the attribute name information to obtain a word frequency vector feature, and determining the information type of the attribute value information; If the information type of the attribute value information is a numerical type, feature extraction is performed on the attribute value information to obtain statistical features; If the information type of the attribute value information is a non-numeric type, feature extraction is performed on the attribute value information to obtain word vector features and entropy value features.

5. The method according to claim 1, characterized in that The second classification model includes an embedding layer, a convolution layer, a pooling layer, and a fully connected layer. The second classification model is based on which the target field information is identified according to the field feature information to obtain a second category identification result corresponding to the target field information, including: Based on the embedding layer, mapping processing is performed on the field feature information to obtain a word vector matrix of a preset dimension; Based on the convolution layer, feature extraction is performed on the word vector matrix of the preset dimension to obtain word vector matrix features; Based on the pooling layer, the word vector matrix features are pooled to obtain target word vector matrix features; Based on the fully connected layer, recognition is performed according to the target word vector matrix features to obtain a second category recognition result corresponding to the target field information.

6. The method according to claim 1, characterized in that After the target field information is identified based on the second classification model according to the field feature information to obtain a second category identification result corresponding to the target field information, the method further includes: If it is determined that the second category recognition result is unsuccessful, clustering and category prediction are performed on the target field information based on the field feature information based on a third classification model to obtain a third category recognition result corresponding to the target field information; If it is determined that the third category recognition result is successful, the confidentiality level information corresponding to the target field information is determined according to the third category recognition result, and a data recognition result is generated according to the target field information and its corresponding category and confidentiality level information.

7. The method according to claim 1, characterized in that The method further comprises: If the first category recognition result or the second category recognition result is a successful recognition, the corresponding target data is obtained, and the target data is cleaned to obtain standard data; The second classification model is trained according to the standard data to update the model parameters of the second classification model to obtain a target second classification model.

8. The method according to claim 1, characterized in that: The obtaining of target field information corresponding to the target data includes: Acquire target data, and extract field information from the target data; The format of the field information is standardized to obtain target field information.

9. A network device, characterized in that: The network equipment includes: A processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of the data identification method according to any one of claims 1 to 8 are realized.

10. A storage medium for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the data identification method according to any one of claims 1 to 8.

Images