Data desensitization method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a data desensitization method which comprises the following steps: performing word segmentation processing on a target data source to obtain multiple types of word segmentation results; generating a word frequency vector corresponding to the target data source according to each word segmentation result; wherein the word frequency vector comprises a plurality of dimension vectors, and each dimension vector corresponds to one type of word segmentation results; calculating the similarity between any two dimension vectors in the word frequency vectors, and generating a similarity matrix of the target data source based on the similarities; identifying the similarity matrix by using a preset classification algorithm, and determining the data category of each segmented word in the target data source; and desensitizing the corresponding segmented words according to the data categories to obtain desensitized data. By applying the technical scheme, rapid, efficient, accurate and comprehensive data identification and desensitization can be realized, and the information security is effectively ensured. The invention further provides a data desensitization device, electronic equipment and a computer readable storage medium which all have the technical effects.

Description

Technical Field

[0001] The present application relates to the field of data security technology, and in particular to a data desensitization method, and also to a data desensitization device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the rapid development of Internet information, people are communicating more and more frequently through various information devices, such as using mobile phones to handle business, which generates a large amount of information and massive log data. Information overload makes people unable to pay attention to events that receive little attention, so mining massive data becomes increasingly important.

[0003] However, with the increase in sensitive data dimensions and data types, the desensitizing system is often unable to identify all data types. When the types of data continue to expand, the method of manually binding the desensitized data and desensitizing algorithms through user-specified data desensitization strategies will appear to be very inefficient. In addition, the original data desensitization strategy may not cover all data types, resulting in some sensitive data not being fully desensitized, thereby increasing the risk of information leakage.

[0004] Therefore, how to achieve fast, efficient, accurate and comprehensive data identification and desensitization to ensure information security is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0005] The purpose of this application is to provide a data desensitization method, which can achieve fast, efficient, accurate and comprehensive data identification and desensitization, and effectively ensure information security; another purpose of this application is to provide a data desensitization device, electronic device and computer-readable storage medium, all of which have the above-mentioned beneficial effects.

[0006] In a first aspect, the present application discloses a data desensitization method, comprising:

[0007] Perform word segmentation processing on the target data source to obtain multi-category word segmentation results;

[0008] Generate a word frequency vector corresponding to the target data source according to each of the word segmentation results; wherein the word frequency vector includes multiple dimensional vectors, and each dimensional vector corresponds to a type of word segmentation result;

[0009] Calculate the similarity between any two dimensional vectors in the word frequency vector, and generate a similarity matrix of the target data source based on each of the similarities;

[0010] Using a preset classification algorithm to identify the similarity matrix, and determine the data category of each word in the target data source;

[0011] Desensitizing is performed on corresponding word segments according to the data category to obtain desensitized data.

[0012] Optionally, calculating the similarity between any two dimensional vectors in the word frequency vector includes:

[0013] The similarity calculation formula is used to calculate the similarity between any two dimensional vectors in the word frequency vector; wherein the similarity calculation formula is:

[0014] ;

[0015] in, Represents a dimension vector and dimension vector The similarity between and Represents the dimension vector Middle participle The number of word frequencies and dimension vectors Middle participle The number of word frequencies; and Represents the dimension vector and dimension vector The average word frequency of Represents a dimension vector and dimension vector The word frequency set.

[0016] Optionally, the using a preset classification algorithm to identify the similarity matrix and determine the data category of each word segment in the target data source includes:

[0017] For each matrix element in the similarity matrix, calculating the distance between each training data in the preset training data set and the matrix element;

[0018] Selecting a preset number of target training data with the smallest distance value from the preset training data set;

[0019] Determine the data category to which each of the target training data belongs, and determine the target data category with the highest occurrence frequency among all the data categories;

[0020] The target data category is used as the data category of the segmentation corresponding to the matrix element.

[0021] Optionally, for each matrix element in the similarity matrix, calculating the distance between each training data in a preset training data set and the matrix element includes:

[0022] For each matrix element in the similarity matrix, a distance calculation formula is used to calculate the distance between each training data in the preset training data set and the matrix element; wherein the distance calculation formula is:

[0023] ;

[0024] in, Represents a matrix element vector With the training data vector The distance between Represents a matrix element vector With the training data vector The dimension of The value range is 1 to A positive integer between .

[0025] Optionally, before desensitizing the corresponding word segments according to the data category and obtaining the desensitized data, the method further includes:

[0026] For each participle, feature extraction is performed on the participle to obtain feature information;

[0027] Performing label matching on the feature information to obtain the sensitivity level of the word segmentation;

[0028] Accordingly, desensitizing the corresponding word segments according to the data category to obtain desensitized data includes:

[0029] For each word segment, determine the desensitization rule corresponding to the word segment according to the data category and sensitivity level of the word segment; wherein the desensitization rule includes a combination of one or more of an encryption algorithm, a K anonymity algorithm, an L diversity algorithm, a differential privacy algorithm, and a generalization processing algorithm;

[0030] The word segmentation is desensitized according to the desensitization rule to obtain the desensitized data.

[0031] Optionally, performing word segmentation processing on the target data source to obtain multiple types of word segmentation results includes:

[0032] Using a preset word segmentation tool to perform word segmentation processing on the target data source to obtain multiple types of initial word segmentation results; wherein the preset word segmentation tool includes a regular matching tool and / or a stuttering word segmentation tool;

[0033] For each of the initial word segmentation results, stop words in the initial word segmentation results are determined and filtered out to obtain the word segmentation results.

[0034] Optionally, generating a word frequency vector corresponding to the target data source according to each of the word segmentation results includes:

[0035] For each word segmentation result, performing word frequency statistics on the word segmentation result to obtain a word frequency statistical result;

[0036] Generate a word frequency matrix using each of the word frequency statistics results;

[0037] Perform vector conversion on the word frequency matrix to obtain the word frequency vector corresponding to the target data source.

[0038] In a second aspect, the present application also discloses a data desensitization device, comprising:

[0039] The word segmentation module is used to perform word segmentation processing on the target data source and obtain multiple types of word segmentation results;

[0040] A generating module, used to generate a word frequency vector corresponding to the target data source according to each of the word segmentation results; wherein the word frequency vector includes multiple dimensional vectors, and each dimensional vector corresponds to a type of word segmentation result;

[0041] A calculation module, used for calculating the similarity between any two dimensional vectors in the word frequency vector, and generating a similarity matrix of the target data source based on each of the similarities;

[0042] An identification module, used to identify the similarity matrix using a preset classification algorithm to determine the data category of each word in the target data source;

[0043] The desensitization module is used to perform desensitization processing on the corresponding word segments according to the data category to obtain desensitized data.

[0044] In a third aspect, the present application further discloses an electronic device, comprising:

[0045] Memory for storing computer programs;

[0046] A processor is used to implement the steps of any one of the data desensitization methods described above when executing the computer program.

[0047] In a fourth aspect, the present application further discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any one of the data desensitization methods described above are implemented.

[0048] The present application provides a data desensitization method, comprising: performing word segmentation processing on a target data source to obtain multiple categories of word segmentation results; generating a word frequency vector corresponding to the target data source according to each of the word segmentation results; wherein the word frequency vector includes multiple dimensional vectors, and each dimensional vector corresponds to a category of word segmentation results; calculating the similarity between any two dimensional vectors in the word frequency vector, and generating a similarity matrix of the target data source based on each of the similarities; identifying the similarity matrix using a preset classification algorithm to determine the data category of each word segmentation in the target data source; performing desensitization processing on the corresponding word segmentation according to the data category to obtain desensitized data.

[0049] By applying the technical solution provided in this application, the automatic identification of the data category to which each word segment in the target data source belongs is realized by combining the word frequency similarity calculation and the preset classification algorithm. At the same time, the automatic desensitization of the corresponding word segment is realized according to the data category to which the word segment belongs. It can be seen that this technical solution can effectively reduce manual operations, realize automatic data type identification and data desensitization based on data type, help to achieve fast, efficient, accurate and comprehensive data identification and desensitization, and further ensure data information security.

[0050] The data desensitizing device, electronic device and computer-readable storage medium provided in this application also have the above-mentioned technical effects, which will not be elaborated in this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the drawings required for use in the description of the prior art and the embodiments of the present application are briefly introduced below. Of course, the drawings described below in relation to the embodiments of the present application are only part of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work, and the obtained other drawings also belong to the protection scope of the present application.

[0052] FIG1 is a schematic diagram of a data desensitization method provided in the present application;

[0053] FIG2 is a schematic diagram of a flow chart of another data desensitization method provided in the present application;

[0054] FIG3 is a schematic diagram of the structure of a data desensitization device provided by the present application;

[0055] FIG. 4 is a schematic diagram of the structure of an electronic device provided in the present application. DETAILED DESCRIPTION

[0056] The core of this application is to provide a data desensitization method, which can achieve fast, efficient, accurate and comprehensive data identification and desensitization, and effectively ensure information security; another core of this application is to provide a data desensitization device, electronic device and computer-readable storage medium, all of which have the above-mentioned beneficial effects.

[0057] In order to describe the technical solutions in the embodiments of the present application more clearly and completely, the technical solutions in the embodiments of the present application will be introduced below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0058] An embodiment of the present application provides a data desensitization method.

[0059] Please refer to FIG. 1 , which is a flow chart of a data desensitization method provided in the present application. The data desensitization method may include the following S101 to S105 .

[0060] S101: Perform word segmentation processing on the target data source to obtain multiple categories of word segmentation results.

[0061] This step aims to implement word segmentation processing of the target data source to obtain multiple categories of word segmentation results. The target data source is the source data that needs to be desensitized, which can be text data. Among them, multiple categories of word segmentation results refer to different ways of word segmentation processing for the same target source data, and the word segmentation results corresponding to each word segmentation method can be obtained. For example, for the target data source "database encryption service", the corresponding one category of word segmentation results can be "database / encryption / service", another category of word segmentation results can be "database encryption / service", and another category of word segmentation results can be "database / encryption service", etc. Therefore, for the same target data source, different word segmentation methods can be used to obtain multiple categories of word segmentation results.

[0062] In one embodiment of the present application, the above-mentioned word segmentation processing of the target data source to obtain multiple types of word segmentation results may include:

[0063] Using a preset word segmentation tool to perform word segmentation processing on the target data source to obtain multiple types of initial word segmentation results; wherein the preset word segmentation tool includes a regular matching tool and / or a stuttering word segmentation tool;

[0064] For each initial word segmentation result, stop words in the initial word segmentation result are determined and filtered out to obtain a word segmentation result.

[0065] The embodiment of the present application provides a word segmentation processing method. Specifically, a pre-installed word segmentation tool can be used to perform word segmentation on the target data source to obtain multiple types of initial word segmentation results. Among them, each type of initial word segmentation result contains all the original words in the target data source. However, considering that general texts may involve common words such as "de", "qi", "zai" (function words and prepositions), these words will inevitably be included in each type of initial word segmentation result. However, these words will not be factors affecting data security. Therefore, the method of screening out stop words (stop words refer to certain words or terms that are automatically filtered out before or after processing natural language data or text in information retrieval to save storage space and improve search efficiency. These words or terms are called stop words, and stop words are all manually input and not automatically generated. After generation, a stop word list will be formed for stop word matching to determine the stop words in natural language data or text) can be used to delete them. That is to say, for each type of initial word segmentation result, the stop words in it can be determined and deleted, and the remaining word segmentation is the final word segmentation result corresponding to this type of initial word segmentation result. Thus, more accurate multiple types of word segmentation results of the target data source can be obtained, and the word segmentation processing volume in subsequent steps can be effectively reduced, which helps to achieve more efficient and accurate data desensitization.

[0066] In addition, the embodiment of the present application also provides two preset word segmentation tools, namely the above-mentioned regular expression matching tool and Jieba word segmentation tool. Of course, in actual applications, it is not necessarily necessary to use these two word segmentation tools to implement word segmentation processing. Other types of word segmentation tools can also be used as long as they can implement the word segmentation function. The present application does not make any limitations in this regard.

[0067] S102: Generate a word frequency vector corresponding to the target data source according to each word segmentation result; where the word frequency vector includes multiple dimensional vectors, and each dimensional vector corresponds to one type of word segmentation result.

[0068] This step aims to generate the word frequency vector of the target data source. Specifically, each type of word segmentation result can generate a dimensional word frequency vector. On this basis, multiple types of word segmentation results can generate multiple dimensional word frequency vectors. Thus, all the dimensional word frequency vectors can be combined into the overall word frequency vector corresponding to the target data source. It can be understood that this word frequency vector is used to identify the data type to which the word segmentation belongs by combining word frequency similarity calculation, so as to facilitate data desensitization processing based on the data type.

[0069] In an embodiment of the present application, the above-mentioned generating a word frequency vector corresponding to the target data source according to each word segmentation result may include:

[0070] For each word segmentation result, perform word frequency statistics on the word segmentation result to obtain a word frequency statistics result;

[0071] Generate a word frequency matrix using the word frequency statistics results;

[0072] Perform vector conversion on the word frequency matrix to obtain the word frequency vector corresponding to the target data source.

[0073] The embodiment of the present application provides a method for generating a word frequency vector corresponding to a target data source. Specifically, for each type of word segmentation result, a word frequency statistics can be performed to obtain the corresponding word frequency statistics results, so that the overall word frequency matrix corresponding to the target data source can be generated based on all the word frequency statistics results. Finally, the word frequency matrix corresponding to the target data source can be converted into a word frequency vector corresponding to the target data source by vector conversion.

[0074] S103: Calculate the similarity between any two dimensional vectors in the word frequency vector, and generate a similarity matrix of the target data source based on each similarity.

[0075] This step aims to realize the word frequency similarity calculation, which is implemented based on the word frequency vector corresponding to the target data source. As mentioned above, each category of word segmentation results of the target data source can generate a word frequency vector of one dimension. On this basis, multiple categories of word segmentation results can generate word frequency vectors of multiple dimensions. Therefore, the word frequency vector corresponding to the target data source is composed of multi-dimensional vectors. Based on this, for any two dimensional vectors in the word frequency vector corresponding to the target data source, similarity calculation can be performed on them, so that the similarity matrix corresponding to the target data source can be generated using all vector similarities. It can be understood that the similarity matrix is ​​used to realize the data type recognition of each word segmentation in combination with the preset recognition algorithm in the subsequent steps.

[0076] In one embodiment of the present application, the above-mentioned calculation of the similarity between any two dimensional vectors in the word frequency vector may include:

[0077] The similarity calculation formula is used to calculate the similarity between any two dimensional vectors in the word frequency vector; the similarity calculation formula is:

[0078] ;

[0079] in, Represents a dimension vector and dimension vector The similarity between and Represents the dimension vector Middle participle The number of word frequencies and dimension vectors Middle participle The number of word frequencies; and Represents the dimension vector and dimension vector The average word frequency of Represents a dimension vector and dimension vector The word frequency set.

[0080] The embodiment of the present application provides a method for calculating vector similarity. For any two dimensional vectors in the word frequency vector corresponding to the target data source, the similarity calculation between the two can be implemented according to the above similarity calculation formula.

[0081] S104: Using a preset classification algorithm to identify the similarity matrix, determine the data category of each word segment in the target data source.

[0082] This step aims to realize the automatic identification of the data category to which each word segmentation in the target data source belongs, and this process can be implemented based on a preset classification algorithm. Specifically, the preset classification algorithm can be used to identify the similarity matrix of the target data source, and the data category of each word segmentation in the target data source can be obtained. Among them, the preset classification algorithm can be any type of machine learning algorithm, such as Kmeans algorithm, K neighbor algorithm, etc., and its specific type does not affect the implementation of this technical solution. It can be customized by technical personnel according to actual needs, and this application does not limit this.

[0083] In one embodiment of the present application, the above-mentioned identification of the similarity matrix by using a preset classification algorithm to determine the data category of each word segment in the target data source may include:

[0084] For each matrix element in the similarity matrix, calculate the distance between each training data in the preset training data set and the matrix element;

[0085] Selecting a preset number of target training data with the smallest distance value from the preset training data set;

[0086] Determine the data category to which each target training data belongs, and determine the target data category with the highest frequency among all data categories;

[0087] The target data category is taken as the data category of the segmentation corresponding to the matrix element.

[0088] The embodiment of the present application provides a specific type of preset classification algorithm, namely the K-nearest neighbor algorithm. Specifically, in the process of realizing data category recognition based on the K-nearest neighbor algorithm, first, for each matrix element in the similarity matrix, the distance between each training data in the preset training data set and the matrix element can be calculated, and the preset number (K) of training data with the closest distance can be selected as the target training data, then, the data category to which each target training data belongs is determined, and the data category with the highest frequency (most occurrences) (target data category) is the data category of the segmentation corresponding to the current matrix element. In this way, data type recognition based on the K-nearest neighbor algorithm is realized.

[0089] In one embodiment of the present application, for each matrix element in the similarity matrix, calculating the distance between each training data in the preset training data set and the matrix element may include:

[0090] For each matrix element in the similarity matrix, the distance calculation formula is used to calculate the distance between each training data in the preset training data set and the matrix element; wherein the distance calculation formula is:

[0091] ;

[0092] in, Represents a matrix element vector With the training data vector The distance between Represents a matrix element vector With the training data vector The dimension of The value range is 1 to A positive integer between .

[0093] The embodiment of the present application provides a method for calculating the distance between training data and matrix elements, namely the above-mentioned distance calculation formula, which is the Euclidean distance calculation formula.

[0094] S105: Desensitizing the corresponding word segments according to the data category to obtain desensitized data.

[0095] This step aims to achieve word segmentation desensitization based on data categories, thereby achieving data desensitization of the target data source. Specifically, for each word segmentation, after determining the data category to which it belongs, it can be desensitized according to its data category to obtain the corresponding desensitized data. More specifically, different desensitization strategies can be pre-set for different data categories, thereby achieving word segmentation desensitization based on data types.

[0096] It can be seen that the data desensitization method provided in the embodiment of the present application realizes the automatic identification of the data category to which each word segment in the target data source belongs by combining the word frequency similarity calculation and the preset classification algorithm, and realizes the automatic desensitization of the corresponding word segment according to the data category to which the word segment belongs. It can be seen that the present technical solution can effectively reduce manual operations, realize automatic data type identification and data desensitization based on data type, help to realize fast, efficient, accurate and comprehensive data identification and desensitization, and further ensure the security of data information.

[0097] Based on the above embodiments:

[0098] In one embodiment of the present application, before desensitizing the corresponding word segments according to the data category and obtaining the desensitized data, the following steps may also be performed: for each word segment, performing feature extraction on the word segment to obtain feature information; performing label matching on the feature information to obtain the sensitivity level of the word segment;

[0099] Correspondingly, the above-mentioned desensitizing the corresponding word segments according to the data category to obtain desensitized data may include: for each word segment, determining the desensitization rule corresponding to the word segment according to the data category and sensitivity level of the word segment; wherein the desensitization rule includes a combination of one or more of an encryption algorithm, a K anonymity algorithm, an L diversity algorithm, a differential privacy algorithm, and a generalization processing algorithm; desensitizing the word segment according to the desensitization rule to obtain desensitized data.

[0100] The data desensitization method provided in the embodiment of the present application can further implement data desensitization processing based on data type and data sensitivity level, so as to further improve the accuracy of data desensitization, and thus ensure data information security. Specifically, before desensitizing the corresponding word segmentation according to the data category and obtaining the desensitized data, for each word segmentation, feature extraction can be performed on it, and the sensitivity level of the analysis can be determined by matching feature labels. Thus, the desensitization rule corresponding to the word segmentation can be determined according to the data category and sensitivity level of the word segmentation, so as to realize the desensitization processing of the current word segmentation using the desensitization rule. Among them, different data categories and different data sensitivity levels are pre-set with different desensitization rules for realizing word segmentation desensitization processing based on data type and data sensitivity level.

[0101] In addition, the embodiments of the present application also provide a variety of desensitization rules, namely, a combination of one or more of the above-mentioned positive encryption algorithm, K anonymity algorithm, L diversity algorithm, differential privacy algorithm, generalization processing algorithm, etc. Of course, in practical applications, it is not necessary to use these types of word segmentation tools to achieve data desensitization processing. Other types of data desensitization algorithms can also be used to achieve the data desensitization function. This application does not limit this.

[0102] Based on the above embodiments, the embodiment of the present application provides another data desensitization method.

[0103] Please refer to FIG. 2 for a flow chart of another data desensitization method provided by the present application, and the implementation process is as follows:

[0104] Step 1: For the data in the target data source, use the regular matching tool and / or the Jieba word segmentation tool to perform word segmentation processing, filter stop words, then merge the word segments, and count the word segments and word frequencies.

[0105] Step 2: construct the word frequency matrix corresponding to the target data source and further convert it into a word frequency vector.

[0106] Step 3, use the Pearson Correlation Coefficient to calculate the similarity between any two dimensional vectors in the word frequency vector:

[0107] ;

[0108] in, Represents a dimension vector and dimension vector The similarity between and Represents the dimension vector Middle participle The number of word frequencies and dimension vectors Middle participle The number of word frequencies; and Represents the dimension vector and dimension vector The average word frequency of Represents a dimension vector and dimension vector The word frequency set.

[0109] Step 4, obtain the similarity matrix from step 3, and use the K nearest neighbor algorithm to classify each word segmentation; wherein, the distance calculation in the K nearest neighbor algorithm can be implemented by the Euclidean distance calculation formula:

[0110] ;

[0111] in, Represents the matrix element vector in the similarity matrix The training data vector of the training data set in the K nearest neighbor algorithm The distance between Represents a matrix element vector With the training data vector The dimension of The value range is 1 to A positive integer between .

[0112] Step 5: Rate the sensitivity of each word segment through feature tag matching.

[0113] Step 6: According to different sensitivity levels and data categories, different desensitization rules are used to desensitize the corresponding word segments; the desensitization rules include encryption algorithms, generalization processing algorithms, K-anonymity algorithms, differential privacy algorithms, etc.

[0114] For example, please refer to Table 1, which is a desensitization rule comparison table provided in this application:

[0115] Table 1 A comparison table of desensitization rules

[0116]

[0117] It can be seen that the data desensitization method provided in the embodiment of the present application realizes the automatic identification of the data category to which each word segment in the target data source belongs by combining the word frequency similarity calculation and the preset classification algorithm, and realizes the automatic desensitization of the corresponding word segment according to the data category to which the word segment belongs. It can be seen that the present technical solution can effectively reduce manual operations, realize automatic data type identification and data desensitization based on data type, help to realize fast, efficient, accurate and comprehensive data identification and desensitization, and further ensure the security of data information.

[0118] An embodiment of the present application provides a data desensitization device.

[0119] Please refer to FIG3, which is a schematic diagram of the structure of a data desensitization device provided by the present application. The data desensitization device may include:

[0120] The word segmentation module 1 is used to perform word segmentation processing on the target data source to obtain multiple types of word segmentation results;

[0121] Generating module 2, used to generate a word frequency vector corresponding to the target data source according to each word segmentation result; wherein the word frequency vector includes multiple dimension vectors, and each dimension vector corresponds to a type of word segmentation result;

[0122] Calculation module 3, used to calculate the similarity between any two dimensional vectors in the word frequency vector, and generate a similarity matrix of the target data source based on each similarity;

[0123] Identification module 4, used to identify the similarity matrix using a preset classification algorithm to determine the data category of each word in the target data source;

[0124] The desensitization module 5 is used to perform desensitization processing on corresponding word segments according to data categories to obtain desensitized data.

[0125] It can be seen that the data desensitization device provided in the embodiment of the present application realizes the automatic identification of the data category to which each word segment in the target data source belongs by combining the word frequency similarity calculation and the preset classification algorithm, and realizes the automatic desensitization of the corresponding word segment according to the data category to which the word segment belongs. It can be seen that the present technical solution can effectively reduce manual operations, realize automatic data type identification and data desensitization based on data type, help to realize fast, efficient, accurate and comprehensive data identification and desensitization, and further ensure the security of data information.

[0126] In one embodiment of the present application, the calculation module 3 can be specifically used to calculate the similarity between any two dimensional vectors in the word frequency vector using a similarity calculation formula; wherein the similarity calculation formula is:

[0127] ;

[0128] in, Represents a dimension vector and dimension vector The similarity between and Represents the dimension vector Middle participle The number of word frequencies and dimension vectors Middle participle The number of word frequencies; and Represents the dimension vector and dimension vector The average word frequency of Represents a dimension vector and dimension vector The word frequency set.

[0129] In one embodiment of the present application, the identification module 4 may include:

[0130] A calculation unit, used for calculating, for each matrix element in the similarity matrix, the distance between each training data in the preset training data set and the matrix element;

[0131] A screening unit, used to screen a preset number of target training data with the smallest distance value in a preset training data set;

[0132] A determination unit, used to determine the data category to which each target training data belongs, and to determine the target data category with the highest occurrence frequency among all data categories;

[0133] The setting unit is used to set the target data category as the data category of the segmentation corresponding to the matrix element.

[0134] In one embodiment of the present application, the above-mentioned calculation unit can be specifically used to calculate the distance between each training data in the preset training data set and the matrix element for each matrix element in the similarity matrix using a distance calculation formula; wherein the distance calculation formula is:

[0135] ;

[0136] in, Represents a matrix element vector With the training data vector The distance between Represents a matrix element vector With the training data vector The dimension of The value range is 1 to A positive integer between .

[0137] In one embodiment of the present application, the data desensitization device may further include a level identification module, which is used to extract features of each segmented word to obtain feature information before desensitizing the corresponding segmented words according to the data category to obtain desensitized data; perform label matching on the feature information to obtain the sensitivity level of the segmented word;

[0138] Correspondingly, the above-mentioned desensitization module 5 can be specifically used to determine the desensitization rules corresponding to each word segmentation according to the data category and sensitivity level of the word segmentation; wherein the desensitization rules include a combination of one or more of the encryption algorithm, K anonymity algorithm, L diversity algorithm, differential privacy algorithm, and generalization processing algorithm; the word segmentation is desensitized according to the desensitization rules to obtain desensitized data.

[0139] In one embodiment of the present application, the above-mentioned word segmentation module 1 can be specifically used to use a preset word segmentation tool to perform word segmentation processing on the target data source to obtain multiple categories of initial word segmentation results; wherein the preset word segmentation tool includes a regular matching tool and / or a stuttering word segmentation tool; for each initial word segmentation result, determine and filter out stop words in the initial word segmentation result to obtain a word segmentation result.

[0140] In one embodiment of the present application, the above-mentioned generation module 2 can be specifically used to perform word frequency statistics on each word segmentation result to obtain a word frequency statistical result; use each word frequency statistical result to generate a word frequency matrix; perform vector conversion on the word frequency matrix to obtain a word frequency vector corresponding to the target data source.

[0141] For an introduction to the device provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0142] An embodiment of the present application provides an electronic device.

[0143] Please refer to FIG. 4 , which is a schematic diagram of the structure of an electronic device provided by the present application. The electronic device may include:

[0144] Memory for storing computer programs;

[0145] A processor, which is used to implement the steps of any of the above-mentioned data desensitization methods when executing a computer program.

[0146] As shown in Fig. 4, it is a schematic diagram of the composition structure of an electronic device, which may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through the communication bus 13.

[0147] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0148] The processor 10 can call the program stored in the memory 11. Specifically, the processor 10 can execute the operations in the embodiment of the data desensitization method.

[0149] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0150] Perform word segmentation processing on the target data source to obtain multi-category word segmentation results;

[0151] Generate a word frequency vector corresponding to the target data source according to each word segmentation result; wherein the word frequency vector includes multiple dimensional vectors, and each dimensional vector corresponds to a type of word segmentation result;

[0152] Calculate the similarity between any two dimensional vectors in the word frequency vector, and generate a similarity matrix of the target data source based on each similarity;

[0153] Use a preset classification algorithm to identify the similarity matrix and determine the data category of each word in the target data source;

[0154] Desensitize the corresponding word segments according to the data category to obtain desensitized data.

[0155] In a possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0156] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0157] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0158] Of course, it should be noted that the structure shown in FIG. 4 does not constitute a limitation on the electronic device in the embodiment of the present application. In actual applications, the electronic device may include more or fewer components than those shown in FIG. 4 , or combine certain components.

[0159] An embodiment of the present application provides a computer-readable storage medium.

[0160] The computer-readable storage medium provided in the embodiment of the present application stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned data desensitization methods can be implemented.

[0161] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0162] For an introduction to the computer-readable storage medium provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0163] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0164] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0165] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0166] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples herein, and the description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A data desensitization method, characterized in that: include: Perform word segmentation processing on the target data source to obtain multi-category word segmentation results; Generate a word frequency vector corresponding to the target data source according to each of the word segmentation results; wherein the word frequency vector includes multiple dimensional vectors, and each dimensional vector corresponds to a type of word segmentation result; Calculate the similarity between any two dimensional vectors in the word frequency vector, and generate a similarity matrix of the target data source based on each of the similarities; Using a preset classification algorithm to identify the similarity matrix, and determine the data category of each word in the target data source; Desensitizing is performed on corresponding word segments according to the data category to obtain desensitized data.

2. The data desensitization method according to claim 1, characterized in that: The calculating the similarity between any two dimensional vectors in the word frequency vector includes: The similarity calculation formula is used to calculate the similarity between any two dimensional vectors in the word frequency vector; wherein the similarity calculation formula is: ； in, Represents a dimension vector and dimension vector The similarity between and Represents the dimension vector Middle participle The number of word frequencies and dimension vectors Middle participle The number of word frequencies; and Represents the dimension vector and dimension vector The average word frequency of Represents a dimension vector and dimension vector The word frequency set.

3. The data desensitization method according to claim 1, characterized in that: The using a preset classification algorithm to identify the similarity matrix and determine the data category of each word segment in the target data source includes: For each matrix element in the similarity matrix, calculating the distance between each training data in the preset training data set and the matrix element; Selecting a preset number of target training data with the smallest distance value from the preset training data set; Determine the data category to which each of the target training data belongs, and determine the target data category with the highest occurrence frequency among all the data categories; The target data category is used as the data category of the segmentation corresponding to the matrix element.

4. The data desensitization method according to claim 3, characterized in that: For each matrix element in the similarity matrix, calculating the distance between each training data in a preset training data set and the matrix element includes: For each matrix element in the similarity matrix, a distance calculation formula is used to calculate the distance between each training data in the preset training data set and the matrix element; wherein the distance calculation formula is: ； in, Represents a matrix element vector With the training data vector The distance between Represents a matrix element vector With the training data vector The dimension of The value range is 1 to A positive integer between .

5. The data desensitization method according to any one of claims 1 to 4, characterized in that: Before desensitizing the corresponding word segments according to the data category and obtaining the desensitized data, the method further includes: For each participle, feature extraction is performed on the participle to obtain feature information; Performing label matching on the feature information to obtain the sensitivity level of the word segmentation; Accordingly, desensitizing the corresponding word segments according to the data category to obtain desensitized data includes: For each word segment, determine the desensitization rule corresponding to the word segment according to the data category and sensitivity level of the word segment; wherein the desensitization rule includes a combination of one or more of an encryption algorithm, a K anonymity algorithm, an L diversity algorithm, a differential privacy algorithm, and a generalization processing algorithm; The word segmentation is desensitized according to the desensitization rule to obtain the desensitized data.

6. The data desensitization method according to claim 1, characterized in that: The word segmentation processing is performed on the target data source to obtain multiple types of word segmentation results, including: Using a preset word segmentation tool to perform word segmentation processing on the target data source to obtain multiple types of initial word segmentation results; wherein the preset word segmentation tool includes a regular matching tool and / or a stuttering word segmentation tool; For each of the initial word segmentation results, stop words in the initial word segmentation results are determined and filtered out to obtain the word segmentation results.

7. The data desensitization method according to claim 1, characterized in that: Generating a word frequency vector corresponding to the target data source according to each of the word segmentation results includes: For each word segmentation result, performing word frequency statistics on the word segmentation result to obtain a word frequency statistical result; Generate a word frequency matrix using each of the word frequency statistics results; Perform vector conversion on the word frequency matrix to obtain the word frequency vector corresponding to the target data source.

8. A data desensitization device, characterized in that: include: The word segmentation module is used to perform word segmentation processing on the target data source and obtain multiple types of word segmentation results; A generating module, used to generate a word frequency vector corresponding to the target data source according to each of the word segmentation results; wherein the word frequency vector includes multiple dimensional vectors, and each dimensional vector corresponds to a type of word segmentation result; A calculation module, used for calculating the similarity between any two dimensional vectors in the word frequency vector, and generating a similarity matrix of the target data source based on each of the similarities; An identification module, used to identify the similarity matrix using a preset classification algorithm to determine the data category of each word in the target data source; The desensitization module is used to perform desensitization processing on the corresponding word segments according to the data category to obtain desensitized data.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, used to implement the steps of the data desensitization method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data desensitization method according to any one of claims 1 to 7 are implemented.

Images