A sensitive data identification method and device, and a storage medium

Abstract

This application discloses a method, apparatus, and storage medium for sensitive data identification. The method specifically includes: an electronic device acquiring a first dataset and a second dataset. The first dataset includes sensitive data with N fields, and the second dataset includes data to be detected with M fields. The first dataset is a pre-stored, labeled sensitive dataset, and the second dataset is a collected, unlabeled dataset. The electronic device merges the data in the first and second datasets with the same character type to obtain a third dataset, which includes data with S fields. The electronic device clusters the data in the third dataset to obtain R classes of data. The electronic device determines the distribution difference between the sensitive data and the data to be detected in the R classes of the third dataset. If the distribution difference is less than a first preset threshold, the data in the third dataset is determined to be sensitive data. This method can improve the efficiency of sensitive data identification.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus and storage medium for identifying sensitive data. Background Technology

[0002] Structured data refers to data stored in databases or spreadsheet text files (e.g., Excel, CSV). Structured data is typically identified by fields, allowing you to determine whether data corresponding to a specific field is sensitive. For example, if a bank's database stores numerical data with a field labeled "amount," then that field can be identified as sensitive user data. However, in some cases, data fields may be missing, necessitating verification of the data itself to determine its sensitivity. Currently, sensitive data identification is primarily performed manually, which is inefficient. Summary of the Invention

[0003] This application provides a method, apparatus, and storage medium for identifying sensitive data, in order to solve the problem of low efficiency in identifying sensitive data.

[0004] Firstly, this application provides a method for identifying sensitive data. This method can be applied to electronic devices with processing capabilities. Specifically, the method includes: the electronic device acquiring a first dataset and a second dataset. The first dataset includes sensitive data with N fields, and the second dataset includes data to be detected with M fields. The first dataset is a pre-stored, labeled sensitive dataset, and the second dataset is a collected, unlabeled dataset. M and N are both positive integers. The electronic device merges the data in the first and second datasets with the same character type to obtain a third dataset, which includes data in S fields, where S is a positive integer. The electronic device clusters the data in the third dataset to obtain R classes of data, where R is a positive integer. The electronic device determines the distribution difference between the sensitive data and the data to be detected in the R classes of the third dataset. If the distribution difference is less than a first preset threshold, the data in the third dataset is determined to be sensitive data.

[0005] In this embodiment, by merging the M fields in the second dataset that have the same character type (e.g., floating-point numbers, letters, etc.) as the N fields in the first dataset, the fields in the second dataset can be initially screened, deleting fields that are clearly not sensitive data, thus obtaining the third dataset. The electronic device further clusters the data in the third dataset to obtain R-class data. Based on the difference in the distribution of sensitive data and data to be detected in the R-class data, it can be determined whether the data in the third dataset is sensitive data. When identifying sensitive data, merging the first and second datasets can delete data that is clearly not sensitive, improving identification efficiency. Further clustering the merged third dataset reveals the distribution of sensitive data and data to be detected, ultimately yielding the identification result. Identifying sensitive data based on distribution is highly efficient.

[0006] Optionally, the electronic device merges data from fields with the same character type in the first and second datasets. This includes: sampling the first field from the N fields of the first dataset and the second field from the M fields of the second dataset, obtaining multiple first sample data and multiple second sample data. The electronic device determines whether the multiple first sample data and multiple second sample data have the same character type. If the character types are the same, the electronic device merges the data from the first field and the data from the second field, and the merged data becomes the data of any field in the third dataset.

[0007] In this embodiment, sampling is used to compare whether the character types of two fields in the first dataset and the second dataset are the same. If they are the same, the data from the two fields are merged into the data of one field in the third dataset. Sampling can improve the efficiency of merging.

[0008] Optionally, the electronic device determines whether the character types of the multiple first sampled data and the multiple second sampled data are the same, including: the electronic device determines the statistical parameters of the multiple first sampled data and the multiple second sampled data respectively, the statistical parameters including mean and / or variance. The electronic device determines whether the character types of the multiple first sampled data and the multiple second sampled data are the same based on whether the difference between the statistical parameters of the multiple first sampled data and the multiple second sampled data is less than a second preset threshold.

[0009] In this embodiment of the application, since the electronic device has a certain degree of accuracy in judging the character type, when judging whether the character types of multiple first sample data and multiple second sample data are the same, it can be judged by the statistical parameters of the sample data. If the difference of the statistical parameters is less than the second preset threshold, it can be considered that the statistical distribution of multiple first sample data and multiple second sample data is approximately the same, and it can be determined that the character types of multiple first sample data and multiple second sample data are the same.

[0010] Optionally, the electronic device determines the distribution difference between sensitive data and data to be detected in the third dataset across class R data, including: the electronic device determining a first set based on the proportion of sensitive data in each class of data within class R data, the first set representing the distribution of sensitive data in class R data; the electronic device determining a second set based on the proportion of data to be detected in each class of data within class R data, the second set representing the distribution of data to be detected in class R data; and the electronic device determining the Euclidean distance between the first set and the second set as the distribution difference between sensitive data and data to be detected in class R data.

[0011] In this embodiment, by clustering the data in the third dataset, a first set corresponding to sensitive data and a second set of data to be detected are obtained. Whether the data to be detected in the third dataset is sensitive data can be determined based on the Euclidean distance between the first and second sets, which is highly efficient.

[0012] Optionally, before clustering the data in the third dataset, the method further includes: converting the character data in the third dataset into numerical data.

[0013] In this embodiment of the application, converting character data in the third dataset into numerical data can improve the processing efficiency of electronic devices.

[0014] Secondly, this application provides a sensitive data identification device. The device includes an acquisition module, a merging module, a clustering module, and a determination module. The acquisition module acquires a first dataset and a second dataset. The first dataset includes sensitive data with N fields, and the second dataset includes data to be detected with M fields. The first dataset is a pre-stored labeled sensitive dataset, and the second dataset is a collected unlabeled dataset. M and N are both positive integers. The merging module merges the data in the first and second datasets with the same character type to obtain a third dataset, which includes data in S fields, where S is a positive integer. The clustering module clusters the data in the third dataset to obtain R classes of data, where R is a positive integer. The determination module determines the distribution difference between the sensitive data and the data to be detected in the R classes of the third dataset. If the distribution difference is less than a first preset threshold, the data in the third dataset is determined to be sensitive data.

[0015] Optionally, the merging module is specifically used to: sample the first field from the N fields of the first dataset and the second field from the M fields of the second dataset, respectively, to obtain multiple first sample data and multiple second sample data. It then determines whether the character types of the multiple first sample data and the multiple second sample data are the same. If the character types are the same, the data from the first field and the data from the second field are merged, and the merged data becomes the data of any field in the third dataset.

[0016] Optionally, the merging module is specifically used to: determine the statistical parameters of multiple first sample data and multiple second sample data respectively, the statistical parameters including mean and / or variance; and determine whether the character types of the multiple first sample data and multiple second sample data are the same based on whether the difference in the statistical parameters of the multiple first sample data and multiple second sample data is less than a second preset threshold.

[0017] Optionally, the determining module is specifically used to: determine a first set based on the proportion of sensitive data in each class of data within the R-class data, the first set being used to characterize the distribution of sensitive data in the R-class data. Determine a second set based on the proportion of the data to be detected in each class of data within the R-class data, the second set being used to characterize the distribution of the data to be detected in the R-class data. Determine the Euclidean distance between the first set and the second set as the difference in distribution between the sensitive data and the data to be detected in the R-class data.

[0018] Optionally, the clustering module is also used to convert character data in a third dataset into numerical data.

[0019] Thirdly, embodiments of this application provide an electronic device including a processor and a memory communicatively connected to the processor. The memory stores computer-executable instructions, which are executed by the processor to enable the processor to perform the method described in any one of the first aspects above.

[0020] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, cause the processor to perform the method described in any one of the first aspects.

[0021] Fifthly, embodiments of this application provide a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor can read the computer program from the computer-readable storage medium, and when the processor executes the computer program, it can implement the method described in any one of the first aspects above. Attached Figure Description

[0022] Figure 1 A flowchart illustrating the sensitive data identification method provided in this application embodiment;

[0023] Figure 2 This is a schematic diagram of the structure of the sensitive data identification device provided in the embodiments of this application;

[0024] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] To better understand the solutions provided by the embodiments of the present invention, some technical concepts involved in the embodiments of the present invention will first be introduced. It should be noted that in the technical solutions of this application, the collection, dissemination, and use of data all comply with the requirements of relevant national laws and regulations.

[0026] As mentioned earlier, structured data in a database table can be identified by its fields to determine whether the data is sensitive. However, in some cases, data fields may be missing, requiring identification based on the data itself. This will be illustrated using Tables 1 and 2 below as examples.

[0027] Table 1

[0028]

[0029] Table 2

[0030]

[0031] Table 1 above shows the normal storage of data in a database table. As you can see, the data fields in Table 1 are clear, and you can determine whether the data is sensitive based on the fields. For example, the "Name" field includes the user's name, which contains the sensitive data "XXX". Similarly, the data under the "Amount" field is sensitive due to user privacy concerns. You can quickly determine whether the data is sensitive by looking at the corresponding fields. As shown in Table 2, when data fields are missing (indicated by "?"), it's impossible to directly determine whether the data is sensitive. For example, the meaning of the data "4256.27", "54212.65", and "1684.74" in Table 2 cannot be intuitively determined.

[0032] A simpler approach is to identify each data point manually, but this method becomes inefficient when dealing with large datasets. While some machine learning algorithms exist for sensitive data identification, these algorithms also require the data to be identified to be input into the model for individual processing, resulting in low efficiency.

[0033] Therefore, this application provides a sensitive data identification method. By merging the data to be detected with sensitive data and clustering the merged data, it is possible to determine whether the data to be detected is sensitive data based on the distribution differences between the clustered data and the sensitive data. Since the identification of sensitive data is based on the distribution of the data, the identification efficiency is high.

[0034] Please see Figure 1 The diagram shows a flowchart of the sensitive data identification method provided in the embodiments of this application. Figure 1 The process illustrated uses an electronic device to perform sensitive data identification as an example. The electronic device can be any device with data storage capabilities; for example, it can be a terminal such as a smartphone, tablet, or desktop computer.

[0035] For ease of description, in the following description, we will distinguish between datasets with clearly defined fields and datasets with missing fields. The datasets with clearly defined fields will be referred to as the first dataset, as shown in Table 1. The datasets with missing fields will be referred to as the second dataset, as shown in Table 2. That is to say, we need to identify the data in the second dataset. It should be understood that the terms "first" and "second" are used only to distinguish between datasets and do not limit the size, content, order, time sequence, priority, or importance of the datasets.

[0036] S101. The electronic device acquires a first dataset and a second dataset. The first dataset includes sensitive data in N fields, and the second dataset includes data to be detected in M ​​fields. The first dataset is a pre-stored labeled sensitive dataset, and the second dataset is a collected unlabeled dataset. M and N are both positive integers.

[0037] The electronic device acquires a first dataset and a second dataset. The data in the first dataset consists of clearly labeled sensitive data. The first dataset can be a pre-collected dataset by the electronic device. The electronic device can manually identify and label the sensitive data fields in the pre-collected dataset. After labeling the first dataset, the electronic device can obtain sensitive data in N fields, where N is a positive integer. The electronic device can pre-store the labeled first dataset.

[0038] The second dataset consists of unlabeled data collected by electronic devices, and some or all fields in this dataset may be missing. Therefore, the data in the second dataset may contain both sensitive and non-sensitive data, requiring the identification of the sensitive data. For ease of description, the data in the second dataset will be referred to as the data to be detected. The second dataset contains M fields, meaning it includes data to be detected across M fields.

[0039] S102. The electronic device merges the data of the same character type fields in the first dataset and the second dataset to obtain the third dataset, which includes data of S fields, where S is a positive integer.

[0040] In this embodiment, since directly identifying itself using data from the second dataset is inefficient, and considering that the first dataset contains known labeled sensitive data, it can be used to identify the second dataset. The first dataset includes data in N fields, and the second dataset includes data to be detected in M ​​fields. All N fields in the first dataset are sensitive data, while not all M fields in the second dataset are sensitive data. The character types of the data in the second dataset may be completely different from those in the first dataset, or they may be the same but not sensitive. Therefore, when identifying the data to be detected in the second dataset, the electronic device can disregard fields that are clearly not sensitive, reducing the amount of data to be identified and improving detection efficiency. For example, if the second dataset contains data such as "10:34", "09:42", "16:23", etc., under an unlabeled field, the character type of that field can be used to determine that it is "time," and the data under that field is non-sensitive data. For example, if the second dataset contains data such as "SAFGJAIOJIAJDO", "OPQJTEQIOTAD", "POQTEUPODFDAF", etc. under a certain unlabeled field, then the meaning of the data under that field is clearly not sensitive data.

[0041] To identify whether the character types in the second dataset are the same as those in the first dataset, the electronic device can sample and compare one field (e.g., the first field) from the N fields of the first dataset with one field (e.g., the second field) from the M fields of the second dataset.

[0042] An electronic device can sample multiple first sample data from a first dataset and multiple second sample data from a second dataset. If the fields to which the multiple second sample data collected by the electronic device belong are not sensitive data fields, their character types may be completely different from those of the multiple first sample data. For example, the character types of the multiple second sample data may be strings of English letters, such as "SAFGJAIOJIAJDO" and "OPQJTEQIOTAD", while the character types of the multiple first sample data may be floating-point numbers, such as "4256.27" and "54212.65". Therefore, it can be determined that the character types of the multiple first sample data and the multiple second sample data are different. It should be understood that the number of sample data should be within a reasonable range. Too few samples may result in unrepresentative data, while too many samples may affect the processing efficiency of the electronic device. The specific number can be determined according to actual needs, and this application embodiment does not impose specific limitations.

[0043] Specifically, electronic devices can determine whether the character types of multiple first and second sampled data are the same by using statistical parameters, including the mean and / or variance. In other words, the electronic device can calculate the mean and / or variance of multiple first and second sampled data. Since statistical parameters characterize the distribution of data, the more similar the distributions of the first and second sampled data are, the higher the probability that their character types are the same. For example, if multiple first and second sampled data are both "height," then the mean and / or variance of the first and second sampled data calculated by the electronic device should be similar, falling within the range of normal human height. Therefore, the electronic device can determine that the character types are the same. For example, if multiple first-sample data points and multiple second-sample data points are both alphabetic data "name", the electronic device can use word2vec to vectorize these data points. It can then calculate the mean and / or variance of the vectorized first-sample data points and the mean and / or variance of the vectorized second-sample data points. If the values ​​are similar, the electronic device can determine that the character types are the same. On the other hand, if multiple first-sample data points are floating-point numbers and multiple second-sample data points are strings of English letters, then the statistical parameters of the first-sample data points and multiple second-sample data points are significantly different, and the electronic device can determine that the character types are different.

[0044] Since statistical parameters can characterize the approximation of the distributions of multiple first and second sampled data points, even though their distributions may differ, electronic devices can set a second preset threshold for the statistical parameters of these data points. When the difference between the statistical parameters is less than this second preset threshold, their distributions are considered sufficiently similar, and they can be considered to have the same character type. The second preset threshold is a preset threshold for the mean and / or variance of the sampled data. As an example, the second preset threshold for the mean can be 2, and the second preset threshold for the variance can be 3. In other words, if the difference between the means and variances of the multiple first and second sampled data points is less than 2, and the difference in variance is less than 3, they can be considered to have the same character type. Conversely, if the difference between the means and variances of the multiple first and second sampled data points is greater than 2, or the difference in variance is less than 3, they can be considered to have different character types, and the data corresponding to the second field can be disregarded.

[0045] The electronic device can compare each field of the second dataset with each field of the first dataset one by one, merging fields corresponding to data with the same character type. It should be noted that the electronic device determines whether the character types are the same by sampling multiple first and second sample data. During merging, it merges the data from the first field corresponding to the first sample data and the data from the second field corresponding to the second sample data. In other words, it determines whether the character types of two fields are the same through sampling, and then merges all data from both fields. The electronic device merges the data from the first and second fields, and the merged data becomes the data for any field in the third dataset. The electronic device merges the data from N fields in the first dataset with the data from M fields in the second dataset that have the same character type to obtain the third dataset. The number of fields in the third dataset can be denoted as S, where S is a positive integer. It should be understood that not all data in the third dataset is sensitive data; further judgment is required for the data in the third dataset.

[0046] S103. The electronic device clusters the data in the third dataset to obtain R classes of data, where R is a positive integer.

[0047] Electronic devices can use a density-based spatial clustering of applications of noise (DBSCAN) algorithm to cluster data in a third-party dataset. To facilitate understanding of this approach, a brief introduction to the DBSCAN algorithm will follow.

[0048] The DBSCAN algorithm is a density-based clustering algorithm that generally assumes that clusters can be determined by the density of sample distribution. The DBSCAN algorithm requires the selection of a distance metric. For the dataset to be clustered, the distance between any two points reflects the density between them. Distance can be used to determine whether points can cluster into the same class; that is, if the distance between points meets certain conditions, the two points are considered closely related and can be grouped into the same class. The DBSCAN algorithm requires two parameters from the user: one is the radius (EPS), representing the extent of a circular neighborhood centered on a given point P; the other is the minimum number of points (minpts) within the neighborhood centered on point P. If the condition is met—that the number of points within the neighborhood of radius EPS centered on point P is not less than minpts—then point P is called a core point. Points within the neighborhood of radius EPS can be continuously searched from point P until the above condition can no longer be met. The found points are then clustered into the same class, i.e., a cluster.

[0049] Electronic devices can use the DBSCAN algorithm to calculate the distance between samples in a third dataset. By setting the radius EPS, closely connected samples in the third dataset can be identified and grouped into one cluster, thus obtaining a cluster category. By classifying all closely connected samples into different categories, we obtain the final clustering results. Electronic devices can cluster the third dataset using clustering algorithms to obtain R clusters, or R categories. Therefore, it can be considered that clustering the third dataset yields R classes of data. It should be noted that before clustering the data in the third dataset, the electronic device can convert the character-based data in the third dataset into numerical data, which can improve the efficiency of clustering the third dataset.

[0050] S104. The electronic device determines the distribution difference between sensitive data and data to be detected in the R-class data of the third dataset. If the distribution difference is less than the first preset threshold, the data in the third dataset is determined to be sensitive data.

[0051] In the third dataset, one field out of the S fields contains data with the same character type. However, while the character type may be the same, the parameters they represent are not entirely identical. In other words, the third dataset may have merged non-sensitive data from the second dataset, resulting in a situation where one field in the third dataset contains a significant amount of non-sensitive data. For example, in step S102, when merging the first and second datasets, merging floating-point numbers with the same character type in both datasets might result in one field in the third dataset containing data such as "amount," "height," or "weight" from both datasets. Therefore, the data in the third dataset can be further identified by using the R-class data obtained from clustering the third dataset.

[0052] Since the third dataset includes sensitive data from the first dataset and data to be tested from the second dataset, the distribution of sensitive data and data to be tested in the R-class data may differ after clustering the data in the third dataset. For example, if fields from non-sensitive data in the second dataset are merged into the data to be tested, it may result in a larger amount of non-sensitive data in the data to be tested. Non-sensitive data will generate a cluster that is far removed from other data during clustering, i.e., a relatively independent cluster, leading to a significant difference in the distribution between sensitive data and data to be tested. Therefore, electronic devices can determine the difference in the distribution between sensitive data and data to be tested in the R-class data of the third dataset. If the difference is too large, it can be assumed that the data to be tested includes non-sensitive fields, and not all of the data to be tested is sensitive data.

[0053] Since the electronic device clusters the third dataset into R classes of data, the distribution of each class can be determined by the proportion of data in each class within the R classes. The proportion can be represented by a fraction or a percentage. Taking fractions as an example, the electronic device can obtain R fractions representing the proportions, which can be represented by (n1, n2, ..., nr). The set of these R fractions can be used to characterize the distribution of data in the R classes. It should be understood that the sum of all elements in this set should be 1, i.e., Σni = 1.

[0054] The electronic device can determine a first set based on the proportion of sensitive data in each class of data within class R, representing the distribution of sensitive data in class R. This first set can be represented by (a1, a2, ..., ar). Similarly, the electronic device can determine a second set based on the proportion of the data to be detected in each class of data within class R, representing the distribution of the data to be detected in class R. This second set can be represented by (b1, b2, ..., br). The electronic device can use the Euclidean distance between the first and second sets to determine the difference in distribution between sensitive data and the data to be detected within class R. The electronic device can set a first preset threshold to determine whether the difference in distribution between the first and second sets is too large. For example, the electronic device can set the first preset threshold to 4. When the difference in distribution between the first and second sets is greater than or equal to the first preset threshold, it can be considered that the difference between the data to be detected and sensitive data is large, and the data to be detected is not considered sensitive data. Conversely, when the difference in distribution between the first and second sets is less than the first preset threshold, it can be considered that the difference between the data to be detected and sensitive data is sufficiently small, and the data to be detected is considered sensitive data. It should be understood that the embodiments of this application do not limit the specific value of the first preset threshold. The electronic device can set the first preset threshold to be small enough to ensure that the distribution of the data to be detected and the sensitive data are as consistent as possible, so as to ensure the accuracy of identifying the data to be detected as sensitive data.

[0055] Please see Figure 2 Based on the same inventive concept, this application provides a sensitive data identification device 200. The device 200 includes: an acquisition module 201, a merging module 202, a clustering module 203, and a determination module 204. The acquisition module 201 acquires a first dataset and a second dataset. The first dataset includes sensitive data with N fields, and the second dataset includes data to be detected with M fields. The first dataset is a pre-stored labeled sensitive dataset, and the second dataset is a collected unlabeled dataset. M and N are both positive integers. The merging module 202 merges data with the same character type in the first and second datasets to obtain a third dataset, which includes data with S fields, where S is a positive integer. The clustering module 203 clusters the data in the third dataset to obtain R classes of data, where R is a positive integer. The determination module 204 determines the distribution difference between sensitive data and data to be detected in the R classes of the third dataset. If the distribution difference is less than a first preset threshold, the data in the third dataset is determined to be sensitive data.

[0056] Optionally, the merging module 202 is specifically used to: sample the first field from the N fields of the first dataset and the second field from the M fields of the second dataset, respectively, to obtain multiple first sample data and multiple second sample data. It then determines whether the character types of the multiple first sample data and the multiple second sample data are the same. If the character types are the same, the data from the first field and the data from the second field are merged, and the merged data becomes the data of any field in the third dataset.

[0057] Optionally, the merging module 202 is specifically used to: determine the statistical parameters of multiple first sample data and multiple second sample data respectively, the statistical parameters including mean and / or variance; and determine whether the character types of the multiple first sample data and multiple second sample data are the same based on whether the difference in the statistical parameters of the multiple first sample data and multiple second sample data is less than a second preset threshold.

[0058] Optionally, the determining module 204 is specifically used to: determine a first set based on the proportion of sensitive data in each class of data within the R-class data, the first set being used to characterize the distribution of sensitive data in the R-class data; determine a second set based on the proportion of the data to be detected in each class of data within the R-class data, the second set being used to characterize the distribution of the data to be detected in the R-class data; and determine the Euclidean distance between the first set and the second set as the difference in distribution between the sensitive data and the data to be detected in the R-class data.

[0059] Optionally, clustering module 203 is also used to convert character data in the third dataset into numerical data.

[0060] Please see Figure 3 Based on the same inventive concept, this application provides an electronic device 300, which includes at least one processor 301, at least one memory 302, and computer program instructions stored in the memory. When the computer program instructions are executed by the processor, the sensitive data identification method described above is implemented.

[0061] Optionally, the processor 301 may be a central processing unit, an application-specific integrated circuit (ASIC), one or more integrated circuits for controlling program execution, a hardware circuit developed using a field-programmable gate array (FPGA), or a baseband processor.

[0062] Optionally, the read-write lock operating device further includes a memory 302 connected to at least one processor 301. The memory 302 may include read-only memory (ROM), random access memory (RAM), and disk storage. The memory 302 is used to store data required by the processor 301 during operation. There may be one or more memories 302. The memory 302 is used in… Figure 3 It is shown together, but it should be noted that memory 302 is not a mandatory functional module, therefore in Figure 3 It is shown in dashed lines.

[0063] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the aforementioned sensitive data identification method.

[0064] In specific implementation, computer-readable storage media include: Universal Serial Bus flash drive (USB), portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0065] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0066] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0067] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0068] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0069] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: Universal Serial Bus flash disks, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0070] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for identifying sensitive data, characterized in that, include: Obtain a first dataset and a second dataset. The first dataset includes sensitive data with N fields, and the second dataset includes data to be detected with M fields. The first dataset is a pre-stored labeled sensitive dataset, and the second dataset is a collected unlabeled dataset. M and N are both positive integers. The data in the first dataset and the second dataset with the same character type are merged to obtain a third dataset, which includes data in S fields, where S is a positive integer. Cluster the data in the third dataset to obtain R classes of data, where R is a positive integer; A first set is determined based on the proportion of the sensitive data in each type of data in the R-type data. The first set is used to characterize the distribution of the sensitive data in the R-type data. A second set is determined based on the proportion of the data to be detected in each of the R types of data. The second set is used to characterize the distribution of the data to be detected in the R types of data. The Euclidean distance between the first set and the second set is determined as the difference in distribution between the sensitive data and the data to be detected in the R-class data. If the distribution difference is less than the first preset threshold, then the data in the third dataset is determined to be sensitive data.

2. The method as described in claim 1, characterized in that, Merging data in fields with the same character type from the first dataset and the second dataset includes: Sample the first field from the N fields of the first dataset and the second field from the M fields of the second dataset to obtain multiple first sample data and multiple second sample data; Determine whether the character types of the plurality of first sampled data and the plurality of second sampled data are the same; If the character types are the same, the data in the first field and the data in the second field are merged, and the merged data is the data of any field in the third dataset.

3. The method as described in claim 2, characterized in that, Determining whether the character types of the plurality of first sampled data and the plurality of second sampled data are the same includes: Statistical parameters of the plurality of first sample data and the plurality of second sample data are determined respectively, wherein the statistical parameters include mean and / or variance; Whether the character types of the multiple first sampled data and the multiple second sampled data are the same is determined based on whether the difference in the statistical parameters of the multiple first sampled data and the multiple second sampled data is less than a second preset threshold.

4. The method as described in claim 2, characterized in that, Before clustering the data in the third dataset, the method further includes: Convert the character data in the third dataset into numerical data.

5. A sensitive data identification device, characterized in that, include: The acquisition module is used to acquire a first dataset and a second dataset. The first dataset includes sensitive data in N fields, and the second dataset includes data to be detected in M ​​fields. The first dataset is a pre-stored labeled sensitive dataset, and the second dataset is a collected unlabeled dataset. M and N are both positive integers. The merging module is used to merge the data of the same character type fields in the first dataset and the second dataset to obtain a third dataset, which includes data of S fields, where S is a positive integer; The clustering module is used to cluster the data in the third dataset to obtain R classes of data, where R is a positive integer; The determination module is configured to: determine a first set based on the proportion of the sensitive data in each category of the R-type data, wherein the first set is used to characterize the distribution of the sensitive data in the R-type data; determine a second set based on the proportion of the data to be detected in each category of the R-type data, wherein the second set is used to characterize the distribution of the data to be detected in the R-type data; determine the Euclidean distance between the first set and the second set as the distribution difference between the sensitive data and the data to be detected in the R-type data; if the distribution difference is less than a first preset threshold, then determine the data in the third dataset as sensitive data.

6. The apparatus as claimed in claim 5, characterized in that, The merging module is specifically used for: Sample the first field from the N fields of the first dataset and the second field from the M fields of the second dataset to obtain multiple first sample data and multiple second sample data; Determine whether the character types of the plurality of first sampled data and the plurality of second sampled data are the same; If the character types are the same, the data in the first field and the data in the second field are merged, and the merged data is the data of any field in the third dataset.

7. The apparatus as claimed in claim 6, characterized in that, The merging module is specifically used for: Statistical parameters of the plurality of first sample data and the plurality of second sample data are determined respectively, wherein the statistical parameters include mean and / or variance; Whether the character types of the multiple first sampled data and the multiple second sampled data are the same is determined based on whether the difference in the statistical parameters of the multiple first sampled data and the multiple second sampled data is less than a second preset threshold.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-4.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-4.

Images