Data desensitization method and device, electronic equipment and computer readable storage medium
Through incremental slice trough processing and desensitization methods combining word vector model and clustering model, the problem of insufficient desensitization of digital data in the prior art is solved, and a higher level of data security and privacy protection is achieved.
Patent Information
- Application Number
- CN202411348449.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2025-03-14
AI Technical Summary
The existing digital data desensitization technology has insufficient security, which leads to the risk of data leakage and improper use, affecting the effectiveness and trust of data desensitization.
The incremental slice trough processing method is used to divide the desensitization data, and the word vector model and pre-trained clustering model are combined to determine the primary and secondary desensitization rules to improve the nonlinearity and complexity of the desensitization results.
It effectively improves the level of data security and privacy protection, maintains the efficiency of conflict mediation, and provides a more reliable and secure solution for the processing of digital sensitive data.
Smart Images
Figure CN119622784A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing, and in particular to a data desensitization method, device, electronic device and computer-readable storage medium. Background Art
[0002] Data desensitization plays a vital role in all areas of social life. Digital data such as account numbers, passwords, and financial information places high demands on desensitization security and privacy protection. Therefore, providing secure and effective desensitization methods for digital data is particularly important.
[0003] Current digital data desensitization technologies still carry risks in practical applications, such as the possibility of data leakage and improper use. These risks seriously affect the effectiveness and reliability of digital sensitive data desensitization.
[0004] Therefore, providing a safe and effective desensitization method specifically for digital data has become an urgent problem to be solved. Summary of the Invention
[0005] The present invention provides a data desensitization method, apparatus, electronic device, and computer-readable storage medium to at least address the problem of insufficient security in digital data desensitization methods in related technologies.
[0006] In a first aspect, an embodiment of the present invention provides a data desensitization method, the method comprising:
[0007] The data to be anonymized is divided into slots according to the rule that the data length satisfies the increasing rule, and at least one group of slot data is obtained;
[0008] Determine a corresponding initial desensitization rule according to the data length of the slot data, desensitize the slot data according to the initial desensitization rule to obtain desensitized data of each group of the slot data, and concatenate the desensitized data of each group of the slot data to obtain the first desensitized data;
[0009] Using a pre-trained clustering model to determine data labels for each group of the slot data, the data labels include a first label representing a large amount of data and a second label representing a small amount of data;
[0010] Based on the distribution of data labels of all the slotted data, a secondary desensitization rule is determined, and according to the secondary desensitization rule, the first desensitized data and the data to be classified are processed to obtain the second desensitized data.
[0011] Optionally, the data to be desensitized is divided into slots according to the rule that the data length satisfies an increasing rule, to obtain at least one group of slot data, including:
[0012] If the data length of the last group of slot data obtained by the slot processing does not meet the increasing rule, the filling element is determined according to the sum of the elements in the previous slot data;
[0013] Add a set number of padding elements to the last group of slot data until the data length meets the increasing rule.
[0014] Optionally, determining the corresponding initial desensitization rule according to the data length includes:
[0015] The corresponding initial desensitization rule is determined according to the parity of the data length value, and the initial desensitization rule is configured as follows:
[0016] The word vector conversion model is used to convert the slot data into a first vector, where the length of the first vector is L;
[0017] Using the value of the slot data as an index, determining a first word according to a pre-constructed first vocabulary, and converting the first word into a second vector using the word vector conversion model, where the length of the second vector is L;
[0018] Adding values in the same column of the first vector and the second vector to obtain a third vector, and determining a conversion rule for the third vector based on the parity to obtain a conversion vector;
[0019] Using the length value as an index, a second word is determined in a pre-constructed second vocabulary table, and the first letter of the second word, the column values of the conversion vector, and the last letter of the second word are concatenated to obtain the desensitized data of the slot data.
[0020] Optionally, determining a conversion rule of the third vector based on the parity to obtain a conversion vector includes:
[0021] When the data length value is an odd number, using the third vector as the conversion vector; and / or
[0022] When the data length value is an even number, the word vector conversion model is used to convert the length value of the slot data into a fourth vector, and the length of the fourth vector is the same as the data length value; the third vector and the fourth vector are concatenated to obtain the conversion vector.
[0023] Optionally, the length L is configured as:
[0024] If the slot data are all 0, the length is a set multiple of the length of the slot data;
[0025] If the slot data is not all 0, the length is the sum of the value of the slot data and the length of the slot data.
[0026] Optionally, the determining of the data labels of each group of the slot data using a pre-trained clustering model includes:
[0027] Taking the absolute value of each group of the slot data to obtain the corresponding data to be classified;
[0028] A pre-trained clustering model is used to determine, based on the data to be classified, whether the data label of each group of the slot data is the first label or the second label.
[0029] Optionally, determining the secondary desensitization rule based on the distribution of data labels of all the slotted data includes:
[0030] When the data labels of the slotted data are all the first label or the second label, the secondary desensitization rule is configured as follows:
[0031] Sum the values of each element in all the data to be classified, and divide the summation result by a preset value to obtain a quotient and a remainder;
[0032] Determine a third word in the second vocabulary using the quotient as an index, determine a fourth word in the second vocabulary using the remainder as an index, and concatenate the first and last letters of the third and fourth words to obtain a desensitized letter group;
[0033] The second desensitized data is obtained according to the first desensitized data and the desensitized letter group.
[0034] Optionally, determining the secondary desensitization rule based on the distribution of data labels of all the slot data includes:
[0035] When the data labels of the slotted data include the first label and the second label, and the number of the first label and the number of the second label are different, the secondary desensitization rule is configured as follows:
[0036] Synthesizing at least one set of new data using a synthetic enhanced oversampling model, wherein the data labels of the new data and the slotted data satisfy the same number of the first labels and the second labels;
[0037] According to the arrangement order of the at least one group of the new data and the arrangement order of each group of continuous numerical values in the first desensitized data, the same-order data are added together to obtain an updated numerical value of the corresponding arrangement order;
[0038] The continuous numerical values in the first desensitized data are replaced with the updated numerical values to obtain the second desensitized data.
[0039] In a second aspect, an embodiment of the present invention provides a data desensitization device, comprising:
[0040] The slotting module is used to slot the desensitized data according to the rule that the data length meets the increasing rule, and obtain at least one group of slot data;
[0041] a first desensitization module, configured to determine a corresponding first desensitization rule according to the data length of the slot data, desensitize the slot data according to the first desensitization rule, obtain desensitized data of each group of the slot data, and concatenate the desensitized data of each group of the slot data to obtain the first desensitized data;
[0042] a determination module, configured to determine a data label for each group of the slot data using a pre-trained clustering model, wherein the data label includes a first label representing a large amount of data and a second label representing a small amount of data;
[0043] The second desensitization module is used to determine the secondary desensitization rules based on the distribution of data labels of all the slot data, and process the first desensitized data and the data to be classified according to the secondary desensitization rules to obtain the second desensitized data.
[0044] In a third aspect, an embodiment of the present invention provides an electronic device, wherein the electronic device includes:
[0045] Memory,
[0046] processor, and
[0047] A computer program stored on the memory and executable on the processor, wherein the processor implements the data desensitization method of the first aspect when executing the computer program.
[0048] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data desensitization method of the first aspect.
[0049] The data desensitization method, device, electronic device, and computer-readable storage medium provided in the embodiments of the present application have at least the following technical effects.
[0050] The data desensitization method provided in an embodiment of the present invention introduces an incremental slicing and slotting method for desensitization, thereby increasing the nonlinearity of the desensitization result. Furthermore, by combining a word vector model, desensitization processing rules are designed for data with an odd length of not less than 1 and for data with an even length of not less than 2, respectively, thereby improving the nonlinearity and complexity of the desensitization processing result. In this way, the data desensitization method effectively improves data security and privacy protection, while maintaining the efficiency of conflict mediation, providing a more reliable and secure solution for the processing of digital sensitive data.
[0051] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0053] Figure 1 is a flow chart showing a data desensitization method according to an exemplary embodiment;
[0054] Figure 2 is a block diagram of a data desensitization device according to an exemplary embodiment;
[0055] Figure 3 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0056] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.
[0057] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.
[0058] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.
[0059] Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by a person of ordinary skill in the technical field to which this application belongs. The words "one", "a", "the" and the like used in this application do not indicate a limit on quantity and may indicate the singular or plural. The terms "include", "comprise", "have" and any variations thereof used in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units that are inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The word "multiple" used in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.
[0060] Example 1
[0061] In a first aspect, an embodiment of the present invention provides a data desensitization method. Figure 1 is a flow chart of a data desensitization method according to an exemplary embodiment. Figure 1 As shown, the method includes:
[0062] Step S101: Slot the data to be desensitized according to the rule that the data length satisfies the increasing rule, and obtain at least one group of slot data.
[0063] Among them, there is no limitation on the method of obtaining the data to be desensitized and the specific data source. However, the data to be desensitized must be digital data. For example, the professional database of people's livelihood and government affairs consultation and web crawler technology are used to obtain domestic people's livelihood and government affairs consultation data. Since people's livelihood and government affairs consultation data includes digital type data, text type data, and data spliced together by numbers and text, it is necessary to extract the digital data required by the patent of this invention and perform structured integration and splicing to form a digital sensitive data set in the people's livelihood and government affairs scenario, and store it in a digital sensitive database.
[0064] Let's assume that a digitally sensitive dataset contains the following fields: consultation event number (unique identifier for the consultation event, such as the consultation event number), ID number (unique identifier for the consultant, such as the ID number), contact number (such as the contact number), annual income (such as 12,000), provident fund account number, number of children (such as 2), and consultation satisfaction score (with scores from 1 to 5, increasing in satisfaction). This digitally sensitive dataset is stored in a digitally sensitive database.
[0065] In step S101, the data to be desensitized is divided into slots according to the rule that the data length satisfies the increasing rule, that is, the data to be desensitized is divided into slots in an incremental slicing manner. Specifically, each slice cuts out one slot, and the number of elements in each slot must increase in sequence starting from 1. Optionally, the increment step of the number of elements in adjacent slots is a fixed step, such as 1, 2, 3, etc., which can be set according to actual conditions.
[0066] The slotting process facilitates the subsequent desensitization of each group of slot data, and because the slots are divided in an incremental manner, it ensures that different slots must have different attributes in terms of data volume (for example, the data length is an odd or even number). Based on this, different desensitization processes can be carried out based on different attributes, thereby increasing the complexity and nonlinearity of the overall desensitization process.
[0067] Since the length of the data to be desensitized is uncertain, the data length of the last group of slots after the slotting process does not necessarily meet the rule of increasing in sequence. In order to cope with this situation, as an example, step S101 specifically includes:
[0068] Step S1011: If the data length of the last group of slot data obtained by the slot processing does not satisfy the increasing rule, a filling element is determined according to the sum of the elements in the previous slot data.
[0069] Optionally, the preceding slot refers to all the slot data before the last group of slots, or the preceding slot refers to the group of slot data closest to the last group of slots. In step S1011, the filling element is determined based on the sum of the elements in the preceding slot data, specifically: the units digit of the sum of the elements in the preceding slot data is used as the filling element; or the first digit of the sum of the elements in the preceding slot data is used as the filling element. In this way, the determined filling elements are more differentiated, the nonlinearity of the overall desensitization method is increased, and the security of data desensitization is guaranteed.
[0070] Step S1012: Add a set number of padding elements to the last group of slot data until the data length meets the increasing rule. The set number is determined based on the number of missing data bits in the last group of slot data to meet the specific increasing rule.
[0071] For example, assuming that the data to be desensitized is d=4 and requires 1 slice, 1 slice is used to cut out 1 slot, and the slot contains 1 element, that is, 4.
[0072] For example, assuming that the data to be desensitized is d=42 and requires 2 slices, two slots are cut out using the 2 slices. The elements contained in the first slot are concatenated to form 4. The second slot is required to contain 2 elements, so 1 element is missing. Since all elements of all the preceding slots are 4, the fill value in the second slot is 4, which is filled into the tail of 2 in the second slot. Therefore, the elements in the second slot are concatenated to form 24.
[0073] For example, assuming that the data to be desensitized is d=428 and requires 2 slices, then two slots are cut out using the 2 slices. The concatenation result of the elements contained in the first slot is 4, and the concatenation result of the elements contained in the second slot is required to be 28.
[0074] For example, assuming that the data to be desensitized is d=42807 and requires 3 slices, two slots are cut out using 3 slices. The elements contained in the first slot are 4 in series, the elements contained in the second slot are 28 in series, and the third slot is required to contain 3 elements, so 1 element is missing. Since all the elements of all the previous slots are 4, 2, and 8 respectively, the sum is 4+2+8=14. Obviously, the value of the ones place of 14 is 4, so the filling value in the third slot is 4, which is filled into the tail of 07 in the third slot. Therefore, the elements in the third slot are connected in series to form 074.
[0075] Step S102: determine the corresponding initial desensitization rule according to the data length of the slot data, desensitize the slot data according to the initial desensitization rule to obtain desensitized data for each group of slot data, and concatenate the desensitized data for each group of slot data to obtain the first desensitized data.
[0076] As described in step S101, the slotting process performed in an incremental slicing manner has different lengths of data in different groups of slots. Based on this, in step S102, the corresponding initial desensitization rule is determined based on the data length, specifically including: determining the corresponding initial desensitization rule based on the parity of the data length value. In this way, the nonlinearity and complexity of the desensitization processing result are improved, ensuring the safety and effectiveness of the desensitization method.
[0077] The initial desensitization rules are configured as follows:
[0078] The first step is to use the word vector conversion model to convert a group of slot data into the first vector, the length of the first vector is L.
[0079] Optionally, the first vector is obtained using a word vector conversion model such as Roberta, Deberta, Distilbert, or Sentence-bert. There are two different ways to obtain the length of the first vector.
[0080] (1) If the slot data is all 0, the length L is a multiple of the slot data length. "All slot data is 0" means that the slot data is all 0. The slot data length refers to the number of bits in the slot data. The multiple can be 2, 3, 4, etc.
[0081] Taking the setting multiplier as 2 as an example, assuming that the slot data d_1=0, the length of the slot data is 1, and the length of the first vector is 2; assuming that the slot data d_3=000, the length of the slot data is 3, and the length of the first vector is 6.
[0082] (2) If the slot data is not all zero, the length L is the sum of the slot data value and the slot data length. Slot data not all zero means that the slot data includes non-zero digits. The slot data value refers to the data value formed by the slot data, and the slot data length refers to the number of bits in the slot data.
[0083] For example, assuming that the slot data d_3=004, it can be known that the value of the slot data is 4, and the length of the slot data is 3. In this case, the length of the first vector L=4+3=7.
[0084] In the second step, the numerical value of the slot data is used as an index, the first word is determined according to the pre-built first vocabulary, and the word vector conversion model is used to convert the first word into a second vector, and the length of the second vector is L.
[0085] The language involved in the first vocabulary list vocab.txt can be Chinese, English, French, etc., without specific limitation. If the first vocabulary list vocab.txt is a vocabulary list containing all Chinese words, each line contains only one Chinese word. If the first vocabulary list vocab.txt is a vocabulary list containing all English words, each line contains only one English word. If the first vocabulary list vocab.txt is a vocabulary list containing all French words, each line contains only one French word.
[0086] Furthermore, assuming that the index of the first row in the first vocabulary table vocab.txt is 0, the index of the second row is 1, the index of the third row is 2, the index of the fourth row is 3, and so on, the numerical value of the slot data can be used as an index to determine the first word.
[0087] Furthermore, a word vector conversion model such as Roberta, Deberta, Distilbert or Sentence-bert is used to convert the first word into a second vector. During the conversion, the length of the second vector is the same as the length of the first vector.
[0088] For example, when the slot data d_3 = 004, the first vocabulary table vocab.txt contains all Chinese words. At this time, the slot data value is 4. The word corresponding to index 4 in vocab.txt is "happy". The word vector conversion model Deberta converts "happy" into a second vector of length 7.
[0089] Step 3: Add the values in the same column of the first vector and the second vector to obtain a third vector, determine the conversion rule of the third vector based on the parity, and obtain a conversion vector.
[0090] For the first vector and the second vector, the indexes corresponding to the same columns are the same.
[0091] For example, suppose the first vector emb_1 = [1, 12, 4, 35, 8, 11, 2] contains seven numbers, namely 1, 12, 4, 35, 8, 11, and 2, and the corresponding index positions are 0, 1, 2, 3, 4, 5, and 6. The second vector emb_2 = [3, 9, 7, 80, 34, 10, 20] contains seven numbers, namely 3, 9, 7, 80, 34, 10, and 20, and the corresponding index positions are 0, 1, 2, 3, 4, 5, and 6. Then, the number at index 0 in the third vector emb_3 is 1+3=4, the number at index 1 in emb_3 is 12+9=21, the number at index 2 in emb_3 is 4+7=11, the number at index 3 in emb_3 is 35+80=115, the number at index 4 in emb_3 is 8+34=43, the number at index 5 in emb_3 is 11+10=21, and the number at index 6 in emb_3 is 2+20=22. Therefore, the third vector emb_3 = [4, 21, 11, 115, 43, 21, 22].
[0092] For another example, suppose the first vector emb_1 = [2, 31, 0, 5, 40], which contains five numbers: 2, 31, 0, 5, 40, and the corresponding index positions are 0, 1, 2, 3, and 4, respectively. The second vector emb_2 = [3, 2, 37, 8, 24], which contains five numbers: 3, 2, 37, 8, 24, and the corresponding index positions are 0, 1, 2, 3, and 4, respectively. Then, the number at index 0 in the third vector emb_3 is 2 + 3 = 5, the number at index 1 in emb_3 is 31 + 2 = 33, the number at index 2 in emb_3 is 0 + 37 = 37, the number at index 3 in emb_3 is 5 + 8 = 13, and the number at index 4 in emb_3 is 40 + 24 = 64. Therefore, the third vector emb_3 = [5, 33, 37, 13, 64].
[0093] After obtaining the third vector, the processing method for the third vector is differentiated based on the parity of the data length of the slot data, thereby increasing the nonlinearity of the desensitization method. The specific example is as follows:
[0094] When the length of the slotted data is an odd number, the third vector is used as the conversion vector. In other words, the third vector is not processed. For example, the third vector emb_3 = [4, 21, 11, 115, 43, 21, 22] obtained above is directly used as the conversion vector.
[0095] When the length of the slot data is an even number, the word vector conversion model is used to convert the slot data value into a fourth vector whose length is the length of the slot data. The third and fourth vectors are concatenated to obtain a conversion vector.
[0096] For example, if the slot data d_2 = 03 and the length L is 2, the word embedding conversion model Deberta converts the slot data d_2 = 03 into a vector emb_4 of length 2. Assume that emb_4 = [23, 31]. Assume that the third vector emb_3 = [5, 33, 37, 13, 64], then the third vector emb_3 is connected in series with the fourth vector emb_4 to obtain the converted vector emb_5 = [5, 33, 37, 13, 64, 23, 31].
[0097] Step 4: Use the length of the slot data as an index to determine the second word in the pre-built second vocabulary table vocab_2.txt. Concatenate the first letter of the second word, the column values of the conversion vector, and the last letter of the second word to obtain the desensitized data of the grouped data.
[0098] The second vocabulary table, vocab_2.txt, refers to alphabetic languages, such as English, French, Spanish, and Arabic. Furthermore, each line in the second vocabulary table contains only one English word. Furthermore, let the index of the first line in vocab_2.txt be 0, the index of the second line be 1, the index of the third line be 2, the index of the fourth line be 3, and so on. In this way, the second word can be determined by using the length of the slot data as an index.
[0099] The values of each column of the transformation vector are obtained by concatenating the values corresponding to each index position in the transformation vector.
[0100] As an example, the process of step 4 is as follows:
[0101] The length of the slot data is L = 7. The numbers at each index position in the conversion vector are concatenated to obtain join_1 = 42111115432122. At this time, L = 7 is used as the index in the second vocabulary table vocab_2.txt. The corresponding word is "city". The first letter c and the last letter t of the word are taken and concatenated to the front and back of join_1 = 42111115432122, respectively, to obtain the desensitized data corresponding to the slot data (i.e., c42111115y).
[0102] As another example, the process for step 4 is as follows:
[0103] The length of the slot data is L = 5. The numbers at each index position in the conversion vector are concatenated to obtain join_1 = 5333713642331. At this time, L = 5 is used as the index in the second vocabulary table vocab_2.txt. The corresponding word is "handle". The first letter h and the last letter e of the word are taken and concatenated to the front and back of join_1 = 5333713642331, respectively, to obtain the desensitized data corresponding to the slot data (that is, h5333713642331e).
[0104] It should also be noted that in step S102, the desensitization results corresponding to each group of slot data need to be connected in series to obtain the first desensitized data. For example, for the first group of slots, the number of elements is 1, which is an odd number, so the desensitization processing rule of odd numbers is applied, and the desensitized data obtained is recorded as t_1. For the second group of slots, the number of elements is 2, which is an even number, so the desensitization processing rule of even numbers is applied, and the desensitized data obtained is recorded as t_2. For the third slot, the number of elements is 3, which is an odd number, so the desensitization rule of odd numbers is applied, and the desensitized data obtained is t_3. In step S102, t_1, t_2, and t_3 are connected in series, that is, t_1t_2t_3, to obtain the first desensitized data. For example, assuming t_1 = v42113115y, t_2 = b82891115m, t_3 = p5333713642991y, then the first desensitized data is g1 = t_1t_2t_3 = v42113115yb82891115mp5333713642991y.
[0105] Step S103: using a pre-trained clustering model to determine data labels for each group of slot data, the data labels include a first label representing a large amount of data and a second label representing a small amount of data.
[0106] The clustering model can be any one of Optics, Dbscan, GMM, Kmeans or Kmeans++. Before step SS103, the process of training the clustering model is as follows:
[0107] For each group of slot data, it is easy to know the column in which the slot data is located in the digital sensitive database (described in step S101). That is, the corresponding field to which the data belongs when it is recorded in the database. Collect all the data in the column, because all the data in the column are composed of numbers, but there is a situation where the number 0 is at the first position of the data, so it is necessary to adopt an absolute value processing method to remove the 0 at the first position of the data. All the collected data will be used as a training set for the clustering model. Therefore, by taking the absolute value processing method, data normalization is achieved, which avoids the interference of data types on the training and prediction of the clustering model, and improves the clustering accuracy of the trained model.
[0108] For example, if the three data in the column named "Consulting Event Number" in a sensitive digital database are 005, 023, and 300, the results obtained after absolute value processing are 5, 23, and 300, respectively. All the data obtained after absolute value processing is used as the training set for the Optics, Dbscan, GMM, Kmeans, or Kmeans++ clustering model to train the clustering model.
[0109] The number of clusters in the clustering model is set to 2, which is used to output the first label and the second label. For clusters containing a large amount of data, each data point in the cluster is marked with the first label, for example, the category label is 1; for clusters containing a small amount of data, each data point in the cluster is marked with the second label, for example, the category label is 0; if the clusters contain the same amount of data, each data point in one cluster is marked with one category label, and each data point in another cluster is marked with another category label.
[0110] In one embodiment, step S103 specifically includes:
[0111] The first step is to take the absolute value of each group of slot data to obtain the corresponding data to be classified bk_i_2.
[0112] In the second step, a pre-trained clustering model is used to determine whether the data label of each group of slot data is the first label or the second label according to the data to be classified.
[0113] For example, if bk_1_2 is 0 and the label class obtained by the clustering model Optics is 0, it means that bk_1_2 belongs to a cluster with less data. If bk_2_2 is 2 and the label class obtained by the clustering model Optics is 0, it means that bk_2_2 belongs to a cluster with less data. If bk_3_2 is 300 and the label class obtained by the clustering model Optics is 1, it means that bk_3_2 belongs to a cluster with more data.
[0114] Through clustering processing, labels for data in different slots are obtained, and then secondary desensitization rules can be configured according to the label distribution of different slots to further improve the nonlinearity of the desensitization results.
[0115] Step S104: Determine a secondary desensitization rule based on the distribution of data labels of all slot data, and process the first desensitized data and the data to be classified according to the secondary desensitization rule to obtain the second desensitized data.
[0116] Based on the above, data labels for different slot data can be distributed in various ways. For example, all slot data may have data labels that are either the first label or the second label, or the data labels for slot data may include both the first label and the second label, but in varying quantities. Based on this, different secondary desensitization rules are configured for different label distributions.
[0117] <First Case>
[0118] When the data labels of the slotted data are all first labels or second labels, the secondary desensitization rule is configured as follows:
[0119] The first step is to sum the values of each element in all the data to be classified, and divide the sum by the preset value to obtain the quotient and remainder.
[0120] For example, when all the label categories are 1, assume that all the data to be classified contains bk_1_2 = 0, bk_2_2 = 02, bk_3_2 = 300. The sum of the values at each position is 5. With 5 as the numerator and 26 as the denominator, the quotient after division is 5, and the remainder is 0.
[0121] The second step is to determine the third word in the second vocabulary using the quotient as an index, determine the fourth word in the second vocabulary using the remainder as an index, and concatenate the first letters of the third word and the fourth word to obtain a desensitized letter group.
[0122] For example, if the quotient is 5 and the remainder is 0, the English word corresponding to index 5 in the second vocabulary is boot (the third word), and the English word corresponding to index 0 is quiet (the fourth word). Boot starts with b and ends with t, while quiet starts with q and ends with t. By concatenating them in series, the resulting desensitized letter group is btqt.
[0123] The third step is to obtain the second desensitized data based on the first desensitized data and the desensitized letter group.
[0124] Specifically, the desensitized letter group and the first desensitized data are connected in series. For example, the first desensitized data g1=v42113115yb82891115mp5333713642991y, and the desensitized letter group is btqt, then the second desensitized data is g2=v42113115yb82891115mp5333713642991ybtqt.
[0125] Among them, the implementation method of the desensitized letter group and the connection method of the desensitized letter group and the first desensitized data are not specifically limited, and the above are only examples.
[0126] <Second Case>
[0127] When the data labels of the slotted data include a first label and a second label, and the number of the first label and the second label is different, the secondary desensitization rule is configured as follows:
[0128] In the first step, a synthetic enhanced oversampling model is used to synthesize at least one set of new data, and the data labels of the new data and the slotted data satisfy the same number of first labels and second labels.
[0129] Optionally, a synthetic augmented oversampling model such as SMOTÉ, ADASYN, or ROSE is introduced to synthesize at least one new set of data, and the number of labels for different categories is kept the same by adding new data.
[0130] For example, suppose bk_1_2 = 3, corresponding to the label class 0, bk_2_2 = 5, corresponding to the label class 0, bk_2_3 = 70, corresponding to the label class 0, and bk_2_4 = 500, corresponding to the label class 1. Suppose the synthetic enhanced oversampling model smote is used to synthesize two new data with the label class 1, namely bk_2_s_1 and bk_2_s_2. Suppose bk_2_s_1 = 407 and bk_2_s_2 = 625. This makes the number of data with the label class 0 and the number of data with the label class 1 both 3, achieving class balance.
[0131] The second step is to add the same-rank data according to the arrangement order of at least one group of new data and the arrangement order of each group of continuous numerical values in the first desensitized data to obtain the updated numerical value of the corresponding arrangement order.
[0132] Among them, the arrangement order of at least one group of new data can be selected as the arrangement order of the new data values from small to large, or the arrangement order of the new data values from large to small. According to the acquisition process of the first desensitized data introduced above, the first desensitized data includes numbers and letters, and only the numbers in the first desensitized data are considered here.
[0133] For example, assuming the synthesized new data is bk_2_s_1 = 407, bk_2_s_2 = 625, and the first desensitized data g1 = v42113115yb82891115mp5333713642991y. According to the second step, bk_2_s_1 = 407 is added to the continuous data 42113115 contained in g1 to obtain the updated value 42113522. bk_2_s_2 = 625 is added to the continuous data 82891115 contained in g1 to obtain the updated value 82891740.
[0134] Step 3: Replace the continuous values in the first desensitized data with the updated values to obtain the second desensitized data.
[0135] According to the above second step example, the updated values are 42113522 and 82891740. At this time, the second desensitized data g2 = v42113522yb82891740mp5333713642991y.
[0136] By adopting the above-mentioned secondary desensitization rules, especially the method of adding in sequence in the second step, at least the desensitized data of the first slot data in the first desensitized data can still participate in the addition calculation, increase the randomness of the desensitization processing, and improve the nonlinearity of the final desensitization result.
[0137] In summary, the embodiment of the present invention designs desensitization processing rules for data with an odd length of not less than 1 and data with an even length of not less than 2 respectively by combining the word vector model, thereby improving the nonlinearity and complexity of the desensitization processing results. Desensitization is performed by introducing an incremental slicing and slotting method to increase the nonlinearity of the desensitization result. New data is obtained by introducing clustering and synthetic enhanced oversampling models, and finally the final desensitization result is obtained by fusion means, replacing the traditional data desensitization method. This data desensitization method can improve data security and privacy protection levels, while maintaining the efficiency of conflict mediation, providing a more reliable and secure solution for the processing of digital sensitive data.
[0138] Example 2
[0139] In a second aspect, based on the data desensitization method provided in the first aspect, an embodiment of the present invention further provides a data desensitization device. Figure 2 The figure is a block diagram of a data desensitization device according to an exemplary embodiment.
[0140] like Figure 2 As shown, the data desensitization device includes:
[0141] The slotting module 210 is used to slot the data to be desensitized according to the rule that the data length satisfies the increasing rule, and obtain at least one group of slotted data;
[0142] The first desensitization module 220 is used to determine the corresponding first desensitization rule according to the data length of the slot data, desensitize the slot data according to the first desensitization rule to obtain desensitized data of each group of slot data, and concatenate the desensitized data of each group of slot data to obtain the first desensitized data;
[0143] A determination module 230 is configured to determine a data label for each group of slot data using a pre-trained clustering model, wherein the data label includes a first label representing a large amount of data and a second label representing a small amount of data;
[0144] The second desensitization module 240 is used to determine the secondary desensitization rules based on the distribution of data labels of all slot data, and process the first desensitized data and the data to be classified according to the secondary desensitization rules to obtain the second desensitized data.
[0145] In one example, the slotting module includes:
[0146] A first determining unit is configured to determine a padding element based on the sum of the elements in the preceding slot data if the data length of the last group of slot data obtained by the slot processing does not satisfy the increasing rule;
[0147] The filling unit is used to add a set number of filling elements to the last group of slot data until the data length meets the increasing rule.
[0148] In one example, the first desensitization module is specifically configured to determine the corresponding first desensitization rule according to the parity of the data length value, and the first desensitization rule is configured as follows:
[0149] The word vector conversion model is used to convert the slot data into a first vector, where the length of the first vector is L;
[0150] Using the value of the slot data as an index, determining a first word according to a pre-constructed first vocabulary, and converting the first word into a second vector using the word vector conversion model, where the length of the second vector is L;
[0151] Adding values in the same column of the first vector and the second vector to obtain a third vector, and determining a conversion rule for the third vector based on the parity to obtain a conversion vector;
[0152] Using the length value as an index, a second word is determined in a pre-constructed second vocabulary table, and the first letter of the second word, the column values of the conversion vector, and the last letter of the second word are concatenated to obtain the desensitized data of the slot data.
[0153] In one example, the initial desensitization rule is further referred to as:
[0154] Determining a conversion rule of the third vector based on the parity to obtain a conversion vector includes:
[0155] When the data length value is an odd number, using the third vector as the conversion vector; and / or
[0156] When the data length value is an even number, the word vector conversion model is used to convert the length value of the slot data into a fourth vector, and the length of the fourth vector is the same as the data length value; the third vector and the fourth vector are concatenated to obtain the conversion vector.
[0157] In one example, the length L is configured as:
[0158] If the slot data are all 0, the length is a set multiple of the length of the slot data;
[0159] If the slot data is not all 0, the length is the sum of the value of the slot data and the length of the slot data.
[0160] In one example, the determining module includes:
[0161] an acquisition unit, configured to take an absolute value of each group of the slot data to obtain corresponding data to be classified;
[0162] The second determining unit is configured to use a pre-trained clustering model to determine, according to the data to be classified, whether the data label of each group of the slot data is the first label or the second label.
[0163] In one example, the second desensitization module is specifically configured to configure the second desensitization rule as follows when the data labels of the slot data are all the first label or the second label:
[0164] Sum the values of each element in all the data to be classified, and divide the summation result by a preset value to obtain a quotient and a remainder;
[0165] Determine a third word in the second vocabulary using the quotient as an index, determine a fourth word in the second vocabulary using the remainder as an index, and concatenate the first and last letters of the third and fourth words to obtain a desensitized letter group;
[0166] The second desensitized data is obtained according to the first desensitized data and the desensitized letter group.
[0167] In one example, the secondary desensitization rule is further configured as follows:
[0168] The determining of the secondary desensitization rule based on the distribution of data labels of all the slot data includes:
[0169] When the data labels of the slotted data include the first label and the second label, and the number of the first label and the number of the second label are different, the secondary desensitization rule is configured as follows:
[0170] Synthesizing at least one set of new data using a synthetic enhanced oversampling model, wherein the data labels of the new data and the slotted data satisfy the same number of the first labels and the second labels;
[0171] According to the arrangement order of the at least one group of the new data and the arrangement order of each group of continuous numerical values in the first desensitized data, the same-order data are added together to obtain an updated numerical value of the corresponding arrangement order;
[0172] The continuous numerical values in the first desensitized data are replaced with the updated numerical values to obtain the second desensitized data.
[0173] In summary, the embodiment of the present invention designs desensitization processing rules for data with an odd length of not less than 1 and data with an even length of not less than 2 respectively by combining the word vector model, thereby improving the nonlinearity and complexity of the desensitization processing results. Desensitization is performed by introducing an incremental slicing and slotting method to increase the nonlinearity of the desensitization result. New data is obtained by introducing clustering and synthetic enhanced oversampling models, and finally the final desensitization result is obtained by fusion means, replacing the traditional data desensitization method. This data desensitization method can improve data security and privacy protection levels, while maintaining the efficiency of conflict mediation, providing a more reliable and secure solution for the processing of digital sensitive data.
[0174] It should be noted that the data desensitization device provided in this embodiment is used to implement the above-mentioned implementation mode, and the details that have been explained will not be repeated. As used above, the terms "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can implement the predetermined functions. Although the device described in the above embodiment is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.
[0175] Example 3
[0176] In a third aspect, an embodiment of the present application provides an electronic device, Figure 3 FIG is a block diagram of an electronic device according to an exemplary embodiment. Figure 3 As shown, the electronic device may include a processor 81 and a memory 82 storing computer program instructions.
[0177] Specifically, the processor 81 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0178] Among them, the memory 82 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 82 may be inside or outside the data processing device. In a specific embodiment, the memory 82 is a non-volatile memory. In a specific embodiment, the memory 82 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.
[0179] The memory 82 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 81 .
[0180] The processor 81 implements any one of the data desensitization methods in the above embodiments by reading and executing computer program instructions stored in the memory 82.
[0181] In one embodiment, the electronic device may further include a communication interface 83 and a bus 80. Figure 3 As shown, the processor 81, the memory 82, and the communication interface 83 are connected via a bus 80 and communicate with each other.
[0182] The communication interface 83 is used to enable communication between the various modules, devices, units, and / or devices in the embodiments of the present application. The communication port 83 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.
[0183] The bus 80 includes hardware, software, or both, and couples the components of the electronic device to each other. The bus 80 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Bus 80 may include one or more buses, where appropriate. Although embodiments herein describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.
[0184] Example 4
[0185] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the data desensitization method provided in the first aspect is implemented.
[0186] The readable storage medium may include, but is not limited to, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.
[0187] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of the data desensitization method provided in the first aspect.
[0188] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or entirely on the remote device.
[0189] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0190] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.
Claims
1. A data desensitization method, characterized in that: The method comprises: According to the rule that the data length satisfies the increasing rule, the data to be anonymized is divided into slots to obtain at least one group of slot data; Determine a corresponding initial desensitization rule according to the data length of the slot data, desensitize the slot data according to the initial desensitization rule to obtain desensitized data of each group of the slot data, and concatenate the desensitized data of each group of the slot data to obtain the first desensitized data; Using a pre-trained clustering model to determine the data labels of each group of the slot data, the data labels include a first label representing a large amount of data and a second label representing a small amount of data; Based on the distribution of data labels of all the slot data, a secondary desensitization rule is determined, and according to the secondary desensitization rule, the first desensitized data and the data to be classified are processed to obtain the second desensitized data.
2. The data desensitization method according to claim 1, characterized in that: The data to be desensitized is processed by slotting according to the rule that the data length satisfies the increasing rule, and at least one group of slot data is obtained, including: If the data length of the last group of slot data obtained by the slot processing does not satisfy the increasing rule, the filling element is determined according to the sum of the elements in the previous slot data; Add a set number of padding elements to the last group of slot data until the data length satisfies the increasing rule.
3. The data desensitization method according to claim 1, characterized in that: The method of determining the corresponding initial desensitization rule according to the data length includes: The corresponding initial desensitization rule is determined according to the parity of the data length value, and the initial desensitization rule is configured as: The word vector conversion model is used to convert the slot data into a first vector, where the length of the first vector is L; Using the value of the slot data as an index, determining a first word in a pre-constructed first vocabulary, and converting the first word into a second vector using the word vector conversion model, wherein the length of the second vector is L; Adding values in the same column of the first vector and the second vector to obtain a third vector, and determining a conversion rule of the third vector based on the parity to obtain a conversion vector; The length value is used as an index to determine a second word in a pre-constructed second vocabulary table, and the first letter of the second word, the column values of the conversion vector, and the last letter of the second word are concatenated to obtain the desensitized data of the slot data.
4. The data desensitization method according to claim 3, characterized in that: The step of determining a conversion rule of the third vector based on the parity to obtain a conversion vector includes: When the data length value is an odd number, using the third vector as the conversion vector; and / or When the data length value is an even number, the word vector conversion model is used to convert the length value of the slot data into a fourth vector, and the length of the fourth vector is the same as the data length value; the third vector and the fourth vector are concatenated to obtain the conversion vector.
5. The data desensitization method according to claim 3, characterized in that: The length L is configured as: If the slot data are all 0, the length is a set multiple of the length of the slot data; If the slot data is not all zero, the length is the sum of the value of the slot data and the length of the slot data.
6. The data desensitization method according to claim 1, characterized in that: The method of using a pre-trained clustering model to determine the data label of each group of the slot data includes: Taking the absolute value of each group of the slot data to obtain the corresponding data to be classified; A pre-trained clustering model is used to determine, based on the data to be classified, that the data label of each group of the slot data is the first label or the second label.
7. The data desensitization method according to claim 6, characterized in that: The determining of the secondary desensitization rule based on the distribution of the data labels of all the slot data includes: When the data labels of the slot data are all the first label or the second label, the secondary desensitization rule is configured as: Sum the values of each element in all the data to be classified, and divide the summation result by a preset value to obtain a quotient and a remainder; Determine a third word in the second vocabulary table using the quotient as an index, determine a fourth word in the second vocabulary table using the remainder as an index, and concatenate the first and last letters of the third and fourth words to obtain a desensitized letter group; The second desensitized data is obtained according to the first desensitized data and the desensitized letter group.
8. The data desensitization method according to claim 6, characterized in that: The determining of the secondary desensitization rule based on the distribution of data labels of all the slot data includes: When the data label of the slot data includes the first label and the second label, and the number of the first label and the number of the second label are different, the secondary desensitization rule is configured as follows: Synthesize at least one set of new data using a synthetic enhanced oversampling model, wherein the data labels of the new data and the slotted data satisfy that the number of the first label and the number of the second label are the same; According to the arrangement order of the at least one group of the new data and the arrangement order of each group of continuous values in the first desensitized data, the same-order data are added to obtain updated values of the corresponding arrangement order; The continuous numerical values in the first desensitized data are replaced with the updated numerical values to obtain the second desensitized data.
9. A data desensitization device, characterized in that: The device comprises: A slot division module is used to perform slot division processing on the data to be desensitized based on the rule that the data length satisfies the increasing rule, and obtain at least one group of slot data; A first desensitization module, used for determining a corresponding first desensitization rule according to the data length of the slot data, desensitizing the slot data according to the first desensitization rule to obtain desensitized data of each group of the slot data, and concatenating the desensitized data of each group of the slot data to obtain the first desensitized data; A determination module, used to determine a data label for each group of the slot data using a pre-trained clustering model, wherein the data label includes a first label representing a large amount of data and a second label representing a small amount of data; The second desensitization module is used to determine the secondary desensitization rules based on the distribution of data labels of all the slot data, and process the first desensitized data and the data to be classified according to the secondary desensitization rules to obtain the second desensitized data.
10. An electronic device, characterized in that: The electronic device comprises: Memory, processor, and A computer program stored on the memory and executable on the processor, wherein the processor implements the data desensitization method as claimed in any one of claims 1 to 8 when executing the computer program.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data desensitization method according to any one of claims 1 to 8 is implemented.