Data desensitization method, device, equipment and computer readable storage medium

Abstract

The embodiment of the application provides a data desensitization method, device and equipment and a computer readable storage medium, the method comprises: obtaining text information containing sensitive data; based on a pre-established rule base and an adaptive sensitive data identification model, field data in the text information is identified and classified to obtain a data table; the data table is converted into a data table field matrix, and the field categories in the data table field matrix are determined; according to the field categories in the data table field matrix, the target sensitive level corresponding to the data table is determined; according to the corresponding relationship between the pre-established sensitive level and the desensitization strategy, the desensitization strategy corresponding to the target sensitive level is determined; according to the desensitization strategy corresponding to the target sensitive level, the sensitive data is desensitized. According to the embodiment of the application, the rapid identification and accurate classification of sensitive data are realized, the identification efficiency and accuracy of sensitive data are improved, and the accurate protection of sensitive data is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data security, and particularly relates to a data desensitization method and device, equipment and a computer readable storage medium. BACKGROUND

[0002] In the big data era, there are many privacy risks in data opening and sharing. At present, although data analysis technologies such as data mining and machine learning can be used to mine the potential data value and protect user privacy through many privacy protection algorithms, in actual application scenarios, due to the continuous change of data types and the complex relationship between data, the original metadata of the data set is usually processed by the system developer through code, so that the sensitive attributes of the data set processed by the code in the production environment are difficult to identify, the sensitive attribute data cannot be accurately located, and therefore the sensitive attribute data cannot be processed by using appropriate privacy protection means, so the protection efficiency of sensitive data is low. SUMMARY

[0003] The embodiments of the application provide a data desensitization method, device, equipment and computer readable storage medium, which can quickly identify and accurately classify sensitive data, improve the identification efficiency and accuracy of sensitive data, and realize accurate protection of sensitive data.

[0004] In a first aspect, the embodiments of the application provide a data desensitization method, which comprises: acquiring text information containing sensitive data; identifying and classifying field data in the text information based on a pre-established rule library and an adaptive sensitive data identification model to obtain a data table; converting the data table into a data table field matrix and determining the field categories in the data table field matrix; determining the target sensitive level corresponding to the data table according to the field categories in the data table field matrix; determining the desensitization strategy corresponding to the target sensitive level according to the corresponding relationship between the pre-established sensitive level and the desensitization strategy; and desensitizing the sensitive data according to the desensitization strategy corresponding to the target sensitive level.

[0005] According to an embodiment of the first aspect of the application, before the text information containing sensitive data is acquired, the method further comprises: acquiring training data containing sample sensitive data, the training data comprising a structured data table; defining attribute sensitivity in the structured data table according to entropy in the structured data table and maximum entropy; taking the attribute sensitivity as a data point for cluster analysis, and identifying the attribute sensitivity of the sample sensitive data by using a target clustering algorithm; establishing an association relationship between attributes based on an association rule mining algorithm, and dividing the training data into a sensitive attribute set and a non-sensitive attribute set; substituting the sensitive attribute set and the non-sensitive attribute set into a pre-established machine learning model, and training the machine learning model based on a target classification algorithm to obtain a trained adaptive sensitive data identification model.

[0006] According to any one of the preceding embodiments of the first aspect of the application, the field data in the text information is identified and classified based on the pre-established rule base and the adaptive sensitive data identification model to obtain a data table, specifically comprising: constructing a rule base composed of regular expressions and / or field information, the field information including field categories and field descriptions; identifying the fields in the text information based on the regular expressions and / or field information stored in the rule base; in the case that the fields in the text information can be identified, marking target fields in the text information that match the regular expressions and / or field information; in the case that the fields in the text information cannot be identified, calling the adaptive sensitive data identification model to identify the fields in the text information.

[0007] According to any one of the preceding embodiments of the first aspect of the application, the fields in the text information are identified based on the regular expressions and / or field information stored in the rule base, specifically comprising: sequentially identifying all attribute values corresponding to each column in the text information, and classifying the attribute values based on the regular expressions and / or field information stored in the rule base; counting the classification distribution of all attribute values corresponding to each column that is detected; for any ith column, if the proportion of the number of attribute values corresponding to the category with the highest frequency in the ith column to the total number of all attribute values in the ith column exceeds a first preset threshold, the category of the ith column is marked as the category with the highest frequency, i being a positive integer; if the proportion of the number of attribute values corresponding to the category with the highest frequency in the ith column to the total number of all attribute values in the ith column is less than or equal to the first preset threshold, the category of the ith column is marked as unidentifiable.

[0008] According to any one of the foregoing embodiments of the first aspect of the application, the data table is converted into a data table field matrix, and the field categories in the data table field matrix are determined, specifically including: constructing a data table dimension classification category library; constructing a data table field matrix using one-hot encoding according to the fields in the data table and the fields recorded in the data table dimension classification category library, the data table field matrix including multiple fields and vector attribute values corresponding to each field; inputting the data table field matrix into a pre-trained random forest algorithm model to predict the category of the data table corresponding to the data table field matrix; and determining the target sensitive level of the data table corresponding to the data table field matrix according to the field categories in the data table field matrix, specifically including: determining the target sensitive level of the data table corresponding to the data table field matrix according to the category of the data table corresponding to the data table field matrix.

[0009] According to any one of the foregoing embodiments of the first aspect of the application, the target sensitive level of the data table is determined according to the category of the data table corresponding to the data table field matrix, specifically including: constructing a corresponding relationship between the data table category and the sensitive level; determining at least one sensitive level corresponding to the category of the data table according to the corresponding relationship between the data table category and the sensitive level; and selecting the sensitive level with the highest priority as the target sensitive level of the data table from the at least one sensitive level in accordance with a preset priority arrangement order.

[0010] According to any one of the foregoing embodiments of the first aspect of the application, the desensitization strategy includes selecting a custom desensitization algorithm corresponding to the general string to perform desensitization processing on the general string; and the custom desensitization algorithm includes at least one of the following: partial retention of the general string, partial masking of the general string, partial truncation of the general string, and partial replacement of the general string.

[0011] According to any one of the foregoing embodiments of the first aspect of the application, after the sensitive data is desensitized according to the desensitization strategy corresponding to the target sensitive level, the method further includes: monitoring and evaluating a plurality of application software in the terminal device, calculating a privacy risk value corresponding to each of the plurality of application software; classifying the plurality of application software according to the privacy risk value corresponding to each of the plurality of application software to obtain a classification result of the plurality of application software; and performing a corresponding target security operation according to the classification result of the plurality of application software.

[0012] According to any one of the foregoing embodiments of the first aspect of the application, the plurality of application softwares in the terminal device are monitored and evaluated, and the privacy risk values corresponding to the plurality of application softwares are calculated, specifically including: obtaining application information of the plurality of application softwares, the application information at least including historical information of the application softwares calling user privacy permissions; extracting preset required specific values from the application information and performing normalization processing to obtain an application privacy item vector; setting a user privacy item vector corresponding to the application privacy item vector; calculating a dot product of the application privacy item vector and the user privacy item vector to obtain a privacy risk indicator; establishing a privacy risk value evaluation model according to the privacy risk indicator; and calculating the privacy risk values corresponding to the plurality of application softwares based on the privacy risk value evaluation model.

[0013] In a second aspect, the embodiments of the present application provide a data desensitization apparatus, which comprises: a first acquisition module configured to acquire text information containing sensitive data; an identification module configured to identify and classify field data in the text information based on a pre-established rule base and an adaptive sensitive data identification model to obtain a data table; a first determination module configured to convert the data table into a data table field matrix and determine field categories in the data table field matrix; a second determination module configured to determine a target sensitive level corresponding to the data table according to the field categories in the data table field matrix; a third determination module configured to determine a desensitization strategy corresponding to the target sensitive level according to a pre-established corresponding relationship between sensitive levels and desensitization strategies; and a desensitization module configured to perform desensitization processing on the sensitive data according to the desensitization strategy corresponding to the target sensitive level.

[0014] In a third aspect, the embodiments of the present application provide an electronic device, which comprises a processor, a memory, and a computer program stored on the memory and executable on the processor, and the computer program, when executed by the processor, implements the steps of the data desensitization method provided in the first aspect.

[0015] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of the data desensitization method provided in the first aspect.

[0016] The data desensitization method, device, equipment and computer readable storage medium provided by the embodiments of the present application combine the rule library and the adaptive sensitive data identification model to identify and classify the field data in the text information, obtain a data table, and realize accurate marking of sensitive data. The data table is converted into a data table field matrix, and the field categories in the data table field matrix are determined. According to the field categories in the data table field matrix, the target sensitive level corresponding to the data table is determined. According to the corresponding relationship between the sensitive level and the desensitization strategy established in advance, the desensitization strategy corresponding to the target sensitive level is determined. According to the desensitization strategy corresponding to the target sensitive level, the sensitive data is desensitized, the rapid identification and accurate classification of the sensitive data are realized, the identification efficiency and accuracy of the sensitive data are improved, and the accurate protection of the sensitive data is realized. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced. Those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0018] Figure 1 is a flowchart of a data desensitization method provided by the embodiments of the present application;

[0019] Figure 2 is another flowchart of a data desensitization method provided by the embodiments of the present application;

[0020] Figure 3 is another flowchart of a data desensitization method provided by the embodiments of the present application;

[0021] Figure 4 is a structural diagram of a data desensitization device provided by the embodiments of the present application;

[0022] Figure 5 is a hardware structural diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0023] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application more clear and apparent, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, but not to limit the present application. The present application can be implemented without some of these specific details by those skilled in the art. The following description of the embodiments is only intended to provide a better understanding of the present application by showing examples of the present application.

[0024] It should be noted that, in this article, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the elements defined by the statement "include" do not exclude the presence of other identical elements in the process, method, article or equipment including the elements.

[0025] It should be understood that the term "and / or" used in this article is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in this article generally represents that the front and rear associated objects have an "or" relationship.

[0026] Various modifications and changes can be made to the present application without departing from the spirit or scope thereof, which will be apparent to one skilled in the art. Therefore, the present application is intended to cover the modifications and variations of the present application falling within the scope of the corresponding claims (claimed technical solutions) and their equivalents. It should be noted that the embodiments provided by the present application can be combined with each other without contradiction.

[0027] Before describing the technical solutions provided by the embodiments of the present application, in order to facilitate the understanding of the embodiments of the present application, the present application first specifically describes the problems existing in the related art:

[0028] As described previously, the inventors of the present application found that although the related art can mine a large amount of potential data value through data analysis technologies such as data mining and machine learning, there are still many privacy protection problems to be solved before data sharing. The existing technology protects user privacy through a privacy protection algorithm, which cannot accurately identify sensitive data in a data set after special processing in actual application scenarios where the types of data are constantly changing, the relationship between data is increasingly complex, and the original metadata of the data set is operated and processed by the system developer through code and other means. Therefore, it is not possible to use appropriate privacy protection means to process sensitive data, so that the user's privacy cannot be quickly and effectively protected.

[0029] In view of the above research findings of the inventors, the embodiments of the present application provide a data desensitization method, device, equipment and computer readable storage medium, which can solve the technical problems of low sensitive data recognition accuracy and low user privacy protection efficiency in the related art.

[0030] The data desensitization method provided by the embodiments of the present application will be introduced first.

[0031] Figure 1 is a flowchart of the data desensitization method provided by the embodiments of the present application. As shown in Figure 1 , the method can include the following steps S101-S106.

[0032] S101, obtaining text information containing sensitive data.

[0033] The sensitive data can include, but is not limited to, personal privacy data such as user's name, ID number, phone number, address, bank account, and enterprise privacy data such as enterprise's business situation and IP address list.

[0034] S102, based on a pre-established rule base and an adaptive sensitive data recognition model, the field data in the text information is recognized and classified to obtain a data table.

[0035] For example, the rule base can be established in the form of rules based on regular expressions in advance. The rules can be determined according to expert experience, and the rules can be expressed as regular expressions. The field categories and corresponding field descriptions are recognized based on the regular expressions. The field categories are unique, which can include but are not limited to ID number, phone number or address, and the number of field categories is not limited in the embodiments. The rule base is maintained by a special person, and when new rules appear, the new rules are input into the rule base in the form of manual addition. The field content that is difficult to be recognized by regular expressions can only input the field category and field description, and the regular expression is displayed as empty. The contents in the rule base cannot be deleted or modified by the user.

[0036] The training data containing sample sensitive data is obtained in advance according to the big data platform and the text information input by the user, as a training set, the content of the training set is updated regularly, the training set is substituted into the pre-established machine learning model for adaptive model training, and the final adaptive sensitive data recognition model is obtained.

[0037] According to the regular expression in the rule base, the field content in the text information is recognized, if the field content in the text information cannot be recognized, the adaptive sensitive data recognition model is called to recognize, after the recognition is completed, the field category corresponding to the field is determined according to the rule of the rule base and the field category is marked, after the marking is completed, the new field content in the text information is continued to be recognized according to the rule base, until all the fields are correctly classified, and the marked data table is obtained.

[0038] S103, convert the data table into a data table field matrix, and determine the field category in the data table field matrix.

[0039] The marked data table is converted into a data table field matrix, each data table in the data table field matrix is classified in an artificial manner according to the existing field category in the rule base, and the field category corresponding to each data table is determined, wherein one data table can correspond to multiple field categories.

[0040] S104, according to the field category in the data table field matrix, determine the target sensitive level corresponding to the data table.

[0041] According to the field category corresponding to each data table in the data table field matrix, each data table is classified as a minimum unit, the category of each data table is determined, and according to the mapping relationship between the data table category and the sensitive level, the target sensitive level corresponding to each data table is determined.

[0042] S105, according to the corresponding relationship between the sensitive level and the desensitization strategy established in advance, determine the desensitization strategy corresponding to the target sensitive level.

[0043] According to the corresponding relationship between the sensitive level and the desensitization strategy, the desensitization strategy corresponding to the target sensitive level of each data table is determined.

[0044] S106, according to the desensitization strategy corresponding to the target sensitive level, the sensitive data is desensitized.

[0045] The desensitization strategy corresponding to the target sensitive level of each data table is executed, and the sensitive data in each data table is desensitized according to the desensitization strategy.

[0046] The above is a specific implementation manner of the data desensitization method provided by the embodiment of the present application. The embodiment of the present application combines the rule base and the adaptive sensitive data identification model to identify and classify the field data in the text information, obtain a data table, and realize accurate marking of sensitive data. The data table is converted into a data table field matrix, and the field categories in the data table field matrix are determined. According to the field categories in the data table field matrix, the target sensitive level corresponding to the data table is determined. According to the corresponding relationship between the sensitive level and the desensitization strategy established in advance, the desensitization strategy corresponding to the target sensitive level is determined. According to the desensitization strategy corresponding to the target sensitive level, the sensitive data is desensitized, the rapid identification and accurate classification of sensitive data are realized, the identification efficiency and accuracy of sensitive data are improved, and the accurate protection of sensitive data is realized.

[0047] Figure 2 is another flowchart of the data desensitization method provided by the embodiment of the present application. As shown in Figure 2 , according to some embodiments of the present application, before S101, the data desensitization method provided by the embodiment of the present application can also include the following steps S201 to S205.

[0048] S201, obtain training data containing sample sensitive data, the training data including a structured data table.

[0049] According to the big data platform and the text information input by the user, obtain training data containing sample sensitive data, the training data including a structured data table.

[0050] S202, define the attribute sensitivity in the structured data table according to the entropy and the maximum entropy in the structured data table.

[0051] S203, use the attribute sensitivity as a data point for cluster analysis, and identify the attribute sensitivity of the sample sensitive data by using a target clustering algorithm.

[0052] The attribute sensitivity of the sample sensitive data is identified by using a target clustering algorithm (k-means clustering algorithm). First, the sensitivity of the information entropy is quantified, and the information entropy of the information source is calculated by using formula (1),

[0053]

[0054] wherein H(X) represents the information entropy of the information source, that is, the mathematical expectation of the self-information amount of each discrete information of the information source as the average information amount of the information source, p(a i ) represents the probability of occurrence of each discrete information, 0≤p(a i )≤1, and satisfies Secondly, according to the information entropy and the maximum discrete entropy, the attribute sensitivity of the sensitive data is calculated by using formula (2),

[0055]

[0056] wherein H(x i ) represents the information entropy, H max (x i ) represents the maximum discrete entropy, SV i represents the attribute sensitivity, and 0<SV i <1, the smaller the value of SV i is, the greater the information entropy of the attribute is, the closer the distance between the attribute and the maximum discrete entropy is, and the more sensitive the attribute is.

[0057] S204, the association relationship between the attributes is established based on the association rule mining algorithm, and the training data is divided into a sensitive attribute set and a non-sensitive attribute set.

[0058] The association relationship between the sensitive attribute set and the non-sensitive attribute set is established by using the association rule mining algorithm (Apriori algorithm), and the sample sensitive data is divided into the sensitive attribute set or the non-sensitive attribute set according to the size of the attribute sensitivity of the sensitive data. For example, by using PAS i →SA k each association rule is marked, and the data set is divided into a sensitive attribute set SA and a non-sensitive attribute set PAS.

[0059] S205, the sensitive attribute set and the non-sensitive attribute set are substituted into the machine learning model established in advance, and the machine learning model is trained based on the target classification algorithm to obtain a trained adaptive sensitive data recognition model.

[0060] The data in the sensitive attribute set and the non-sensitive attribute set is substituted into the machine learning model established in advance, the machine learning model is trained by using the target classification algorithm (IBk algorithm), and the trained model is saved as an adaptive sensitive data recognition model.

[0061] Through the above embodiment, an efficient recognition process is established by combining the adaptive sensitive data recognition model, the recognition of the text information is quickly completed according to the attributes of the sensitive data, the recognition efficiency and accuracy are improved, and the machine learning model is continuously trained through the storage and regular update of the data, so that the adaptive ability of the model is further enhanced while the recognition efficiency is ensured, thereby more accurate recognition of subsequent different types of sensitive data is realized.

[0062] In one example, based on a pre-established rule base and an adaptive sensitive data identification model, field data in the text information is identified and classified to obtain a data table, specifically including: constructing a rule base composed of regular expressions and / or field information, the field information including field categories and field descriptions; identifying the fields in the text information based on the regular expressions and / or field information stored in the rule base; in the case that the fields in the text information can be identified, marking the target fields in the text information that match the regular expressions and / or field information; in the case that the fields in the text information cannot be identified, calling the adaptive sensitive data identification model to identify the fields in the text information.

[0063] Exemplarily, the rule base constructed stores regular expressions and / or field information, the field information including field categories identified based on the regular expressions and corresponding field descriptions. The fields in the text information are identified based on the regular expressions and / or field information, if the fields in the text information can be identified, the field categories of the target fields matching the regular expressions and / or field information are marked, and if the fields in the text information cannot be identified, the adaptive sensitive data identification model is called for identification.

[0064] Through the above embodiments, the rule base is combined with the adaptive sensitive data identification model to achieve full coverage of sensitive data identification, compared with the traditional identification and classification method, the sensitive data can be identified more deeply, and the identification efficiency of the sensitive data is improved.

[0065] In one example, the fields in the text information are identified based on the regular expressions and / or field information stored in the rule base, specifically including: identifying all attribute values corresponding to each column in the text information in turn, and classifying the attribute values based on the regular expressions and / or field information stored in the rule base; counting the classification distribution of all attribute values corresponding to each column detected; for any i-th column, if the proportion of the number of attribute values corresponding to the category with the highest frequency in the i-th column to the total number of all attribute values in the i-th column exceeds a first preset threshold, the category of the i-th column is marked as the category with the highest frequency, i is a positive integer; if the proportion of the number of attribute values corresponding to the category with the highest frequency in the i-th column to the total number of all attribute values in the i-th column is less than or equal to the first preset threshold, the category of the i-th column is marked as unidentifiable.

[0066] Exemplarily, all attribute values corresponding to each column field in the text information are sequentially identified based on the rule base, and the field categories corresponding to each attribute value are determined, the classification distribution of all attribute values corresponding to each column field being detected is counted, if the proportion of the number of attribute values corresponding to the field category with the highest frequency in a column field to the number of all attribute values corresponding to the column field exceeds a first preset threshold, the field category with the highest frequency is taken as the category of the column field, and if the proportion is less than or equal to the first preset threshold, the category of the column field is marked as unidentifiable.

[0067] In one example, the data table is converted into a data table field matrix, and the field categories in the data table field matrix are determined, specifically including: constructing a data table dimension classification category library; constructing a data table field matrix using one-hot encoding according to the fields recorded in the data table and the data table dimension classification category library, the data table field matrix including multiple fields and vector attribute values corresponding to each field; inputting the data table field matrix into a pre-trained random forest algorithm model to predict the category of the data table corresponding to the data table field matrix; and determining the target sensitive level corresponding to the data table according to the field categories in the data table field matrix, specifically including: determining the target sensitive level corresponding to the data table according to the category of the data table corresponding to the data table field matrix.

[0068] Exemplarily, the data table dimension classification category library is constructed, and the categories of the data table can include but are not limited to personal information data and user package data, etc. A zero vector with a length equal to the total number of field categories is constructed according to the field categories recorded in the rule base, and each field category is corresponded to the zero vector, taking each field category as a feature. If a field corresponding to a field category is included in the data table, the vector attribute value corresponding to the field is marked as 1. The data table is classified by artificial, and the vector attribute values of all field categories corresponding to all data tables are marked as 1, and finally a data table field matrix of n rows and k columns is obtained, where n is the number of field categories recorded in the rule base, and k is the number of data tables.

[0069] Based on the number of data table categories in the data table dimension classification category library, a corresponding number of random forest algorithm models are trained, the data table field matrix is input into the trained random forest algorithm model to predict the category of the data table corresponding to the data table field matrix, and the data table classification result classified in a single data table as the minimum unit is output. If the output data table classification result is incorrect or there is no category label that can be assigned to the data table, the user can correct the data table category, and after the correction is completed, the random forest algorithm model is trained according to the new data table category. After the categories of each data table are determined, the sensitive levels of the data tables are divided according to the categories of the data tables.

[0070] Through the above examples, the user can modify the data table category according to the actual situation, so that the structured data classification and grading method is more flexible and can be applied to various application scenarios.

[0071] In one example, the target sensitive level corresponding to the data table is determined according to the category of the data table corresponding to the data table field matrix, specifically including: constructing a corresponding relationship between the data table category and the sensitive level; determining at least one sensitive level corresponding to the category of the data table according to the corresponding relationship between the data table category and the sensitive level; and selecting the highest priority sensitive level from the at least one sensitive level as the target sensitive level corresponding to the data table according to a preset priority arrangement order.

[0072] For example, the corresponding relationship between the data table category and the sensitive level is constructed, for example, the data table category is personal information data, the sensitive level is the first level, the data table category is user package data, the sensitive level is the second level, the levels are ranked from high to low as the first level, the second level, the third level and the fourth level, the first level represents the highest sensitive level, and the fourth level represents the lowest sensitive level, which is not limited in the embodiment. According to the above corresponding relationship, at least one target sensitive level corresponding to each data table is determined. If it is found that the data table has a higher category of sensitive level during the traversal of the category of each data table, the higher sensitive level is selected as the target sensitive level of the data table.

[0073] In one example, the desensitization strategy includes selecting a custom desensitization algorithm corresponding to the general string to perform desensitization processing on the general string; the custom desensitization algorithm includes at least one of the following: partially retaining the general string, partially masking the general string, partially intercepting the general string, and partially replacing the general string.

[0074] For example, the sensitive data is desensitized according to the desensitization strategy corresponding to the target sensitive level of the data table. The string data is read and split in a specific interval, the strings in the interval are respectively split and simulated to generate new strings, and the intervals are randomly combined to complete the desensitization of the sensitive data.

[0075] Through the above examples, after determining the target sensitive level of the data table, the sensitive data to be protected is desensitized according to the target sensitive level, which reduces the data processing steps while ensuring data security and improves the protection efficiency of the sensitive data.

[0076] Figure 3 is another flowchart of the data desensitization method provided by the embodiment of the application. As Figure 3As shown, according to some embodiments of the present application, optionally, after the sensitive data is desensitized according to the desensitization strategy corresponding to the target sensitive level in S106, the data desensitization method provided by the embodiments of the present application can further include the following steps S301-S303.

[0077] S301, monitoring and evaluating a plurality of application software in a terminal device to calculate a privacy risk value corresponding to each of the plurality of application software.

[0078] S302, classifying the plurality of application software according to the privacy risk value corresponding to each of the plurality of application software to obtain a classification result of the plurality of application software.

[0079] According to the privacy risk value corresponding to each of the plurality of application software, the plurality of application software is classified, for example, the application software is divided into three types of high-risk application, medium-risk application and low-risk application.

[0080] S303, performing a corresponding target security operation according to the classification result of the plurality of application software.

[0081] According to the classification result, each application software is marked differently to remind the user to close the related application permission, and the system intercepts the behavior of the application software to obtain the permission and provides desensitized information, and the user can manually select to add any application software to a trusted application list to avoid automatic interception by the system.

[0082] Through the above embodiments, the risk assessment of the application software is realized, and the function of reminding and preferentially automatically intercepting the user according to the classification result of the application software is achieved, thereby protecting the data privacy and security of the user.

[0083] In one example, monitoring and evaluating a plurality of application software in a terminal device to calculate a privacy risk value corresponding to each of the plurality of application software includes: obtaining application information of the plurality of application software, the application information at least including historical information of the application software calling user privacy permission; extracting a preset required specific value from the application information and performing normalization processing to obtain an application privacy item vector; setting a user privacy item vector corresponding to the application privacy item vector; calculating a dot product of the application privacy item vector and the user privacy item vector to obtain a privacy risk indicator; establishing a privacy risk value evaluation model according to the privacy risk indicator; and calculating the privacy risk value corresponding to each of the plurality of application software based on the privacy risk value evaluation model.

[0084] Exemplarily, information of a plurality of application software and history information of calling user privacy permission thereof are acquired as application information of the application software. A preset required specific value is extracted from the application information and normalized processing is performed to obtain an application privacy item vector V. A user privacy item vector L is set corresponding to the application privacy item vector V, and an expression of the user privacy item vector L is L=(R1, R2, R3, …, Rn). i ), R i is a privacy sensitivity of partial sensitive data, and an expression thereof is m represents a sensitive attribute set, and n represents a non-sensitive attribute set. Dot product of the application privacy item vector V and the user privacy item vector L is calculated to obtain a privacy risk index. A privacy risk value evaluation model is established according to the privacy risk index, and the privacy risk value evaluation model is used to calculate a privacy risk value corresponding to each of the plurality of application software by using formula (3),

[0085]

[0086] wherein, P(y) represents the privacy risk value, w i represents a regression coefficient of a variable, i represents a privacy text number value (0, 1, 2, …, n), and B represents a distance between the application privacy item vector V and the user privacy item vector L.

[0087] Through the above embodiment, an application software monitoring and evaluation algorithm is introduced, the application privacy item vector of the application software is combined with the privacy sensitivity of the user sensitive data, a corresponding privacy risk value evaluation model is constructed, a rule base and a self-adaptive sensitive data recognition model are cooperated, a complete sensitive data protection system is formed, the security of the user sensitive data is improved, and the system is more humanized.

[0088] Based on the data desensitization method provided in the above embodiment, correspondingly, the application also provides a specific implementation manner of a data desensitization device. Please refer to the following embodiments.

[0089] Firstly, referring to Figure 4 , the data desensitization device 40 provided in the embodiment of the application includes the following modules:

[0090] The first acquisition module 401 is configured to acquire text information containing sensitive data.

[0091] The recognition module 402 is configured to recognize and classify the field data in the text information based on a pre-established rule base and a self-adaptive sensitive data recognition model to obtain a data table.

[0092] The first determination module 403 is configured to convert the data table into a data table field matrix and determine a field category in the data table field matrix.

[0093] The second determining module 404 is configured to determine a target sensitive level corresponding to the data table according to the field category in the data table field matrix.

[0094] The third determining module 405 is configured to determine a desensitization strategy corresponding to the target sensitive level according to a pre-established corresponding relationship between the sensitive level and the desensitization strategy.

[0095] The desensitization module 406 is configured to perform desensitization processing on the sensitive data according to the desensitization strategy corresponding to the target sensitive level.

[0096] The data desensitization apparatus provided by the embodiments of the present application combines the rule library and the adaptive sensitive data identification model to identify and classify the field data in the text information, obtains a data table, and realizes accurate marking of sensitive data. The data table is converted into a data table field matrix, and the field category in the data table field matrix is determined. According to the field category in the data table field matrix, the target sensitive level corresponding to the data table is determined. According to the pre-established corresponding relationship between the sensitive level and the desensitization strategy, the desensitization strategy corresponding to the target sensitive level is determined. According to the desensitization strategy corresponding to the target sensitive level, desensitization processing is performed on the sensitive data. The data desensitization apparatus realizes rapid identification and accurate classification of sensitive data, improves the identification efficiency and accuracy of sensitive data, and realizes accurate protection of sensitive data.

[0097] In some embodiments, the data desensitization apparatus 40 can further include a model training module configured to obtain training data containing sample sensitive data, the training data including a structured data table; define attribute sensitivity in the structured data table according to entropy and maximum entropy in the structured data table; use the attribute sensitivity as a data point for cluster analysis, and identify the attribute sensitivity of the sample sensitive data by using a target clustering algorithm; establish an association relationship between attributes based on an association rule mining algorithm, and divide the training data into a sensitive attribute set and a non-sensitive attribute set; substitute the sensitive attribute set and the non-sensitive attribute set into a pre-established machine learning model, and train the machine learning model based on a target classification algorithm to obtain a trained adaptive sensitive data identification model.

[0098] In some embodiments, the identification module 402 is specifically configured to: construct a rule library composed of regular expressions and / or field information, the field information including a field category and a field description; identify the fields in the text information based on the regular expressions and / or the field information stored in the rule library; in a case where the fields in the text information can be identified, mark a target field in the text information that matches the regular expressions and / or the field information; and in a case where the fields in the text information cannot be identified, call the adaptive sensitive data identification model to identify the fields in the text information.

[0099] In some embodiments, the identification module 402 described above can also be configured to: sequentially identify all attribute values corresponding to each column in the text information, and classify the attribute values based on the regular expressions and / or field information stored in the rule library; count the classification distribution of all attribute values corresponding to each column; for any ith column, if the proportion of the number of attribute values corresponding to the class with the highest frequency in the ith column to the total number of all attribute values in the ith column exceeds a first preset threshold, the class of the ith column is marked as the class with the highest frequency, i is a positive integer; if the proportion of the number of attribute values corresponding to the class with the highest frequency in the ith column to the total number of all attribute values in the ith column is less than or equal to the first preset threshold, the class of the ith column is marked as unidentifiable.

[0100] In some embodiments, the first determination module 403 described above is specifically configured to: construct a data table dimension classification category library; construct a data table field matrix using one-hot encoding according to the fields recorded in the data table and the data table dimension classification category library, the data table field matrix including multiple fields and vector attribute values corresponding to each field; input the data table field matrix into a pre-trained random forest algorithm model to predict the category of the data table corresponding to the data table field matrix; and determine the target sensitivity level corresponding to the data table according to the categories of the fields in the data table field matrix, specifically including: determining the target sensitivity level corresponding to the data table according to the category of the data table corresponding to the data table field matrix.

[0101] In some embodiments, the first determination module 403 described above can also be configured to: construct a corresponding relationship between data table categories and sensitivity levels; determine at least one sensitivity level corresponding to the category of the data table according to the corresponding relationship between the data table categories and the sensitivity levels; and select the sensitivity level with the highest priority as the target sensitivity level corresponding to the data table from the at least one sensitivity level in accordance with a preset priority arrangement order.

[0102] In some embodiments, the desensitization strategy includes selecting a custom desensitization algorithm corresponding to the general string to perform desensitization processing on the general string; and the custom desensitization algorithm includes at least one of the following: partial retention of the general string, partial masking of the general string, partial truncation of the general string, and partial replacement of the general string.

[0103] In some embodiments, the data desensitization apparatus 40 can further include a monitoring and evaluation module configured to monitor and evaluate a plurality of application software in a terminal device, calculate a privacy risk value corresponding to each of the plurality of application software, classify the plurality of application software according to the privacy risk value corresponding to each of the plurality of application software to obtain a classification result of the plurality of application software, and perform a corresponding target security operation according to the classification result of the plurality of application software.

[0104] In some embodiments, the monitoring evaluation module is specifically configured to: acquire application information of the plurality of application software, the application information at least including historical information of the application software calling the user privacy permission; extract a preset required specific value from the application information and perform normalization processing to obtain an application privacy item vector; set a user privacy item vector corresponding to the application privacy item vector; calculate a dot product of the application privacy item vector and the user privacy item vector to obtain a privacy risk index; establish a privacy risk value evaluation model according to the privacy risk index; and calculate the respective privacy risk values of the plurality of application software based on the privacy risk value evaluation model.

[0105] Figure 4 Each module / unit in the apparatus has the function of implementing each step in the data desensitization method provided by the method embodiments, and can achieve the corresponding technical effects. For brevity, the description will not be repeated here.

[0106] Based on the data desensitization method provided in the above embodiments, the application also provides a specific implementation of an electronic device. Please refer to the following embodiments.

[0107] Figure 5 A hardware structure schematic diagram of an electronic device provided by an embodiment of the application is shown.

[0108] The electronic device can include a processor 501 and a memory 502 having computer program instructions stored therein.

[0109] Specifically, the processor 501 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement one or more embodiments of the application.

[0110] The memory 502 can include a mass storage for data or instructions. By way of example and not limitation, the memory 502 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In one example, the memory 502 can include a removable or non-removable (or fixed) medium, or the memory 502 is a non-volatile solid state memory. The memory 502 can be internal or external to the integrated gateway disaster recovery device.

[0111] In one example, the memory 502 can be a Read Only Memory (ROM). In one example, the ROM can be a mask programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0112] The memory 502 can include a read only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions that, when executed (e.g., by one or more processors), are operable to perform the operations described with reference to the methods according to an aspect of the present application.

[0113] The processor 501 implements the methods / techniques in the above-described method embodiments by reading and executing computer program instructions stored in the memory 502, and achieves the corresponding technical effects of the method embodiments performing the methods / steps thereof. For brevity, this will not be described again.

[0114] In one example, the electronic device can further include a communication interface 503 and a bus 510. As shown, the processor 501, the memory 502, and the communication interface 503 are connected through the bus 510 and complete communication therebetween. Figure 5

[0115] The communication interface 503 is mainly used to realize the communication between the modules, devices, units, and / or equipment in the embodiments of the present application.

[0116] ​Bus 510 includes hardware, software, or both, to couple electronic devices to each other in a manner that allows information to be passed between or among them. The bus can include, for example, an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand® interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, bus 510 can include one or more buses. Although the application embodiments described and illustrated herein focus on specific buses, the application contemplates any suitable bus or interconnect.

[0117] In addition, in combination with the data desensitization method in the above embodiments, the embodiments of the present application can provide a computer readable storage medium to implement. The computer readable storage medium has computer program instructions stored thereon; the computer program instructions are executed by the processor to implement any one of the data desensitization methods in the above embodiments. Examples of the computer readable storage medium include non-transitory computer readable storage medium, such as electronic circuit, semiconductor memory device, ROM, random access memory, flash memory, erasable ROM (EROM), floppy disk, CD-ROM, optical disc, hard disk.

[0118] It needs to be clear that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the present application.

[0119] The functions noted in the description of the structure block diagrams above can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, and the like. When implemented in software, the elements of the present application are program or code segments that are used to perform the required tasks. The program or code segments can be stored in a machine-readable medium, or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. A "machine-readable medium" includes any medium that can store or transport information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and the like. The code segments can be downloaded via computer networks such as the Internet, intranets, and the like.

[0120] It is also important to note that the examples described herein can be implemented in a variety of systems, including and / or incorporating software, firmware, hardware, fixed logic circuitry, and / or a combination of both. Furthermore, the order of any of the steps, processes, or actions described herein is not essential for the practice of the application unless otherwise specified. That is, where steps, processes, or actions can be performed in other sequences, or omitted, combinations of such steps, processes, or actions can be used, and other steps, processes, or actions can be used.

[0121] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. Alternatively, computer program implemented steps can be

[0122] The above merely describes a specific implementation of the present application. Those skilled in the art can clearly understand the specific working processes of the system, modules and units described above for the convenience and brevity of description, and can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein again. It should be understood that the protection scope of the present application is not limited to this, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A data de-sensitization method, characterized in that, The method comprises the following steps: obtaining text information containing sensitive data; based on a pre-established rule base and an adaptive sensitive data identification model, identifying and classifying the field data in the text information to obtain a data table; converting the data table into a data table field matrix and determining the field categories in the data table field matrix; determining the target sensitive level corresponding to the data table according to the field categories in the data table field matrix; determining the desensitization strategy corresponding to the target sensitive level according to the pre-established corresponding relationship between the sensitive level and the desensitization strategy; performing desensitization processing on the sensitive data according to the desensitization strategy corresponding to the target sensitive level; wherein, based on the pre-established rule base and the adaptive sensitive data identification model, the field data in the text information is identified and classified to obtain a data table, which specifically comprises: constructing the rule base composed of regular expressions and / or field information, wherein the field information includes field categories and field descriptions; identifying the fields in the text information based on the regular expressions and / or field information stored in the rule base; in the case that the fields in the text information can be identified, marking the target fields in the text information that match the regular expressions and / or field information; in the case that the fields in the text information cannot be identified, calling the adaptive sensitive data identification model to identify the fields in the text information, combining the identification process of the adaptive sensitive data identification model, identifying the text information according to the properties of the sensitive data, and identifying different types of sensitive data through data storage and regular updating.

2. The method of claim 1, wherein, Before the text information containing sensitive data is obtained, the method further comprises: obtaining training data containing sample sensitive data, wherein the training data includes structured data tables; defining the attribute sensitivity in the structured data table according to the entropy and maximum entropy in the structured data table; using the attribute sensitivity as the data point of cluster analysis, identifying the attribute sensitivity of the sample sensitive data by using a target clustering algorithm; based on the association rule mining algorithm, establishing the association relationship between attributes, and dividing the training data into a sensitive attribute set and a non-sensitive attribute set; substituting the sensitive attribute set and the non-sensitive attribute set into a pre-established machine learning model, and training the machine learning model based on a target classification algorithm to obtain the trained adaptive sensitive data identification model.

3. The method of claim 1, wherein, The identification of the fields in the text information based on the regular expressions and / or field information stored in the rule base specifically comprises: identifying all attribute values corresponding to each column in the text information in turn, and classifying the attribute values based on the regular expressions and / or field information stored in the rule base; statistically analyzing the classification distribution of all attribute values corresponding to each column that are detected; For any ith column, if the proportion of the number of attribute values corresponding to the category with the highest frequency in the ith column in the total number of attribute values in the ith column exceeds a first preset threshold, the category of the ith column is marked as the category with the highest frequency, i is a positive integer; If the proportion of the number of attribute values corresponding to the category with the highest frequency in the ith column in the total number of attribute values in the ith column is less than or equal to the first preset threshold, the category of the ith column is marked as unidentifiable.

4. The method of claim 1, wherein, The method further comprises: constructing a data table dimension classification category library; constructing the data table field matrix using one-hot encoding according to the fields recorded in the data table and the data table dimension classification category library, the data table field matrix comprising a plurality of fields and vector attribute values corresponding to each field; inputting the data table field matrix into a pre-trained random forest algorithm model to predict the category of the data table corresponding to the data table field matrix; The method further comprises: determining the target sensitivity level corresponding to the data table according to the category of the data table corresponding to the data table field matrix.

5. The method of claim 4, wherein, The method further comprises: constructing a correspondence between data table categories and sensitivity levels; determining at least one sensitivity level corresponding to the category of the data table according to the correspondence between data table categories and sensitivity levels; selecting a sensitivity level with the highest priority as the target sensitivity level corresponding to the data table in a preset priority arrangement order.

6. The method of claim 1, wherein, The desensitization strategy comprises selecting a custom desensitization algorithm corresponding to a general string to perform desensitization processing on the general string. The custom desensitization algorithm comprises at least one of the following: partial retention of the general string, partial masking of the general string, partial truncation of the general string, and partial replacement of the general string.

7. The method of claim 1, wherein, After the sensitive data is desensitized according to the desensitization strategy corresponding to the target sensitivity level, the method further comprises: monitoring and evaluating a plurality of application software in a terminal device to calculate a privacy risk value corresponding to each of the plurality of application software; classifying the plurality of application software according to the privacy risk value corresponding to each of the plurality of application software to obtain a classification result of the plurality of application software; performing a corresponding target security operation according to the classification result of the plurality of application software.

8. The method of claim 7, wherein, The method further comprises: obtaining application information of the plurality of application software, the application information at least comprising historical information of the application software calling user privacy permissions; obtaining an application privacy item vector from the application information; setting a user privacy item vector corresponding to the application privacy item vector; calculating a dot product of the application privacy item vector and the user privacy item vector to obtain a privacy risk indicator; establishing a privacy risk value evaluation model according to the privacy risk indicator; calculating a privacy risk value corresponding to each of the plurality of application software based on the privacy risk value evaluation model.

9. A data de-identification apparatus, comprising: Comprise: The first acquisition module is used for acquiring text information containing sensitive data; The identification module is used for identifying and classifying field data in the text information based on a pre-established rule library and an adaptive sensitive data identification model, to obtain a data table; The first determination module is used for converting the data table into a data table field matrix and determining a field category in the data table field matrix; The second determination module is used for determining a target sensitive level corresponding to the data table according to the field category in the data table field matrix; The third determination module is used for determining a desensitization strategy corresponding to the target sensitive level according to a pre-established corresponding relationship between sensitive levels and desensitization strategies; The desensitization module is used for performing desensitization processing on the sensitive data according to the desensitization strategy corresponding to the target sensitive level; The identification module is further used for constructing the rule library composed of regular expressions and / or field information, the field information including a field category and a field description; identifying a field in the text information based on the regular expressions and / or the field information stored in the rule library; In a case where the field in the text information can be identified, a target field in the text information matching the regular expressions and / or the field information is marked; In a case where the field in the text information cannot be identified, the adaptive sensitive data identification model is called to identify the field in the text information, an identification process is established in combination with the adaptive sensitive data identification model, the text information is identified according to attributes of the sensitive data, and different types of the sensitive data are identified through storage and regular update of data.

10. An electronic device, comprising: The electronic device comprises a processor, a memory, and a computer program stored on the memory and executable on the processor, and the computer program is executed by the processor to implement the steps of the data desensitization method according to any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium, and the computer program is executed by the processor to implement the steps of the data desensitization method according to any one of claims 1 to 8.

Images