Data processing method, device, equipment, storage medium and program product

Abstract

The present application discloses a data processing method, device, equipment, storage medium and program product. The method includes: obtaining a data set to be processed, the data set to be processed includes multiple transactions, each transaction includes at least one data item; using an association rule mining algorithm to mine the data set to be processed, and obtaining frequent item sets and association rules related to the frequent item sets; for each sensitive rule, using an evolutionary multi-objective optimization algorithm and a pre-built multi-objective optimization model, searching for multiple transactions corresponding to the sensitive rule, and generating multiple candidate data cleaning strategies; for each target transaction in the target data cleaning strategy, performing the following operations to obtain a target data set, and the target data cleaning strategy is any one of the multiple candidate data cleaning strategies: deleting at least one data item related to the sensitive rule in the target transaction. According to the embodiment of the present application, the accuracy and quality of the data can be improved while hiding sensitive information.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular, relates to a data processing method, device, equipment, storage medium and program product. Background Art

[0002] Cross-domain data fusion is a technology that aims to break through geographical and organizational boundaries and achieve data sharing and integration. Through cross-domain data fusion technology, different regions and organizations can share data information, thereby promoting optimal resource allocation.

[0003] Although cross-domain data fusion technology has brought significant benefits, it also has the risk of privacy leakage to a certain extent. For example, personal information in user data, such as name, home address, contact information, etc., once leaked, may bring serious privacy risks to individuals. In addition, sensitive knowledge contained in the data, such as frequent patterns, association rules, etc., also has the risk of being improperly used during data sharing. In order to protect this sensitive information, the data content involving sensitive information is usually directly modified or deleted to interfere with the original data.

[0004] However, in the process of interfering with the original data, directly deleting or modifying the data content involving sensitive information may cause certain characteristics of the original data to change, thereby affecting the accuracy and quality of the data. Summary of the invention

[0005] Embodiments of the present invention provide a data processing method, apparatus, device, storage medium, and program product, which can improve the accuracy and quality of data while hiding sensitive information.

[0006] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising:

[0007] Acquire a data set to be processed, where the data set to be processed includes multiple transactions, and each transaction includes at least one data item;

[0008] Using an association rule mining algorithm to mine the processing data set, a frequent item set and association rules related to the frequent item set are obtained, wherein the frequent item set refers to an item set whose support is greater than a preset first support threshold, the item set includes multiple data items, and the association rule includes a rule whose confidence based on the frequent item set screening is greater than a preset first confidence threshold;

[0009] For each sensitive rule, an evolutionary multi-objective optimization algorithm and a pre-built multi-objective optimization model are used to search for multiple transactions corresponding to the sensitive rule to generate multiple candidate data cleaning strategies. The sensitive rule is a rule that needs to be hidden determined from multiple association rules. The candidate data cleaning strategy includes the target transaction to be cleaned. The multi-objective optimization model is constructed with the goal of minimizing multiple decision variables. The multiple decision variables are determined based on multiple parameters determined after data cleaning is performed according to the candidate data cleaning strategy. The multiple decision variables include the reduction in support of frequent itemsets, the change in confidence of non-sensitive rules, the number of modified transactions, and the number of deleted data items.

[0010] For each target transaction in the target data cleaning strategy, perform the following operations to obtain a target data set, where the target data cleaning strategy is any one of multiple candidate data cleaning strategies:

[0011] Delete at least one data item in the target transaction that is related to the sensitivity rule.

[0012] In a second aspect, an embodiment of the present application provides a data processing device, the device comprising:

[0013] An acquisition module is used to acquire a data set to be processed, where the data set to be processed includes multiple transactions, and each transaction includes at least one data item;

[0014] A determination module is used to mine the processing data set using an association rule mining algorithm to obtain frequent item sets and association rules related to the frequent item sets, wherein the frequent item sets refer to item sets whose support is greater than a preset first support threshold, the item sets include multiple data items, and the association rules include rules whose confidence based on the frequent item set screening is greater than a preset first confidence threshold;

[0015] A generation module is used to search for multiple transactions corresponding to each sensitive rule using an evolutionary multi-objective optimization algorithm and a pre-built multi-objective optimization model, and generate multiple candidate data cleaning strategies, wherein the sensitive rule is a rule that needs to be hidden and is determined from multiple association rules. The candidate data cleaning strategy includes a target transaction to be cleaned. The multi-objective optimization model is constructed with the goal of minimizing multiple decision variables. The multiple decision variables are determined based on multiple parameters determined after data cleaning is performed according to the candidate data cleaning strategy. The multiple decision variables include a reduction in the support of frequent itemsets, a change in the confidence of non-sensitive rules, the number of modified transactions, and the number of deleted data items.

[0016] The deletion module is used to perform the following operations for each target transaction in the target data cleaning strategy to obtain a target data set, where the target data cleaning strategy is any one of multiple candidate data cleaning strategies: deleting at least one data item related to the sensitive rule in the target transaction.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;

[0018] When the processor executes the computer program instructions, the data processing method as described in any one of the first aspects is implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer readable storage medium is stored computer program instructions, and when the computer program instructions are executed by a processor, the data processing method as described in any one of the first aspects is implemented.

[0020] In a fifth aspect, an embodiment of the present application provides a program product, which includes a computer program, and when the computer program is processed and executed, it implements the data processing method as described in any one of the first aspects.

[0021] The data processing method, device, equipment, storage medium and program product of the embodiments of the present application, through a multi-objective optimization model, comprehensively considers multiple decision variables related to frequent item sets and association rules, and uses an evolutionary multi-objective optimization algorithm to generate multiple candidate data cleaning strategies, which can effectively find the best balance between multiple goals. The target data cleaning strategy calculated by integrating the evolutionary multi-objective optimization algorithm and the multi-objective optimization model can accurately determine the target transaction while meeting multiple optimization goals. Furthermore, the data items of the target transaction are deleted, effectively realizing the side effects of minimizing the usefulness and integrity of the data cleaning operation on the original data while hiding sensitive information.

[0022] Specifically: Minimizing the reduction in support of frequent itemsets can ensure that sensitive information is hidden while reducing the impact on other frequent itemsets and maintaining the integrity of the data set. Minimizing the change in confidence of non-sensitive rules can reduce the impact of data cleaning on non-sensitive association rules and effectively avoid non-sensitive association rules becoming unreliable due to data cleaning. Minimizing the number of modified transactions can reduce the workload and potential errors of data cleaning and effectively maintain the authenticity of the data. Minimizing the number of deleted data items can reduce the impact of data cleaning on the structure of the data set to be processed and maintain the stability of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical strategy of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1A schematic diagram showing a flow chart of a data processing method provided by an embodiment of the present application is shown;

[0025] Figure 2 A flowchart of a specific implementation method of S102 provided in an embodiment of the present application is shown;

[0026] Figure 3 A flowchart of a specific implementation method of S103 provided in an embodiment of the present application is shown;

[0027] Figure 4 A flowchart of a specific implementation method of S104 provided in an embodiment of the present application is shown;

[0028] Figure 5 A schematic diagram showing the structure of a data processing device provided by an embodiment of the present application is shown;

[0029] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0030] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical strategy and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0031] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0032] Before describing the technical strategies provided by the embodiments of the present application, in order to facilitate understanding of the embodiments of the present application, the present application first specifically describes the relevant technologies:

[0033] Cross-domain data fusion is a technology that realizes data sharing across regions and organizational domains, and is a key means to solve data silos. By using data mining or machine learning technology, valuable patterns and regularities can be extracted from shared data, thereby significantly improving the data value of a single data domain. However, despite the potential benefits of data sharing, many data holders or locations are still reluctant to share data. This hesitation mainly stems from concerns about the leakage of sensitive information. Taking the business field as an example, key strategic information or profit models may be hidden in the data. Once this sensitive information is leaked, it may bring significant business risks and loss of interests to the company. Therefore, how to achieve the fusion and utilization of cross-domain data while effectively protecting sensitive data to maximize its potential value has become an urgent problem to be solved.

[0034] There are two main categories of controllable interactions of sensitive information in cross-domain data fusion: the first category is the sensitive information contained in the data itself, such as user name, home address, and contact information. The second category involves sensitive patterns or knowledge behind the data, which are often expressed in the form of frequent patterns, association rules, and sequence patterns. In order to protect this sensitive information, some modifications can be made to the original data, which is also called "data cleaning". The purpose of data cleaning is to make sensitive data no longer statistically significant through appropriate processing of the data, so that it cannot be mined from the cleaned data. However, data modification may not only damage data integrity, but also lead to the loss of non-sensitive knowledge. For example, some patterns that were originally above the minimum support threshold may no longer meet the threshold after data cleaning, thereby losing valuable information.

[0035] Existing data cleaning methods are often limited to a single dimension, either rules or frequent patterns, and lack consideration of the combined side effects of the two. This single-dimensional evaluation method makes the results overly dependent on the parameters set by the user, such as reducing the false hiding of non-sensitive patterns, avoiding unnecessary loss of non-sensitive association rules, and controlling the generation of false rules. When these parameters change, the effect of data cleaning may vary greatly, thus affecting the accuracy and reliability of the cleaned data.

[0036] In order to solve the problems of the related art, the embodiments of the present application provide a data processing method, device, equipment, storage medium and program product. The data processing method provided by the embodiments of the present application is first introduced below.

[0037] Figure 1 FIG. 1 is a flow chart of a data processing method provided by an embodiment of the present application. Figure 1 As shown, the data processing method may include the following steps: S101~S104.

[0038] S101: Acquire a data set to be processed, where the data set to be processed includes multiple transactions, and each transaction includes at least one data item.

[0039] S102: Using an association rule mining algorithm to mine the processing data set to obtain frequent item sets and association rules related to the frequent item sets, wherein the frequent item sets refer to item sets whose support is greater than a preset first support threshold, and the item sets include multiple data items, and the association rules include rules whose confidence based on the frequent item set screening is greater than a preset first confidence threshold.

[0040] S103: For each sensitive rule, an evolutionary multi-objective optimization algorithm and a pre-built multi-objective optimization model are used to search for multiple transactions corresponding to the sensitive rule to generate multiple candidate data cleaning strategies. The sensitive rule is a rule that needs to be hidden determined from multiple association rules. The candidate data cleaning strategy includes a target transaction to be cleaned. The multi-objective optimization model is constructed with the goal of minimizing multiple decision variables. The multiple decision variables are determined based on multiple parameters determined after data cleaning is performed according to the candidate data cleaning strategy. The multiple decision variables include the reduction in support of frequent itemsets, the change in confidence of non-sensitive rules, the number of modified transactions, and the number of deleted data items.

[0041] S104: For each target transaction in the target data cleaning strategy, perform the following operations to obtain a target data set, where the target data cleaning strategy is any one of the multiple candidate data cleaning strategies:

[0042] Delete at least one data item in the target transaction that is related to the sensitivity rule.

[0043] In an embodiment of the present application, a multi-objective optimization model is used to comprehensively consider multiple decision variables related to frequent item sets and association rules, and an evolutionary multi-objective optimization algorithm is used to generate multiple candidate data cleaning strategies, which can balance various objectives. The target data cleaning strategy calculated by integrating the evolutionary multi-objective optimization algorithm and the multi-objective optimization model can accurately determine the target transaction while meeting multiple optimization objectives. Furthermore, the data items of the target transaction are deleted, effectively realizing the side effects of minimizing the usefulness and integrity of the data cleaning operation on the original data while hiding sensitive information.

[0044] Specifically: Minimizing the reduction in support of frequent itemsets can ensure that sensitive information is hidden while reducing the impact on other frequent itemsets and maintaining the integrity of the data set. Minimizing the change in confidence of non-sensitive rules can reduce the impact of data cleaning on non-sensitive association rules and effectively avoid non-sensitive association rules becoming unreliable due to data cleaning. Minimizing the number of modified transactions can reduce the workload and potential errors of data cleaning and effectively maintain the authenticity of the data. Minimizing the number of deleted data items can reduce the impact of data cleaning on the structure of the data set to be processed and maintain the stability of the data.

[0045] In some embodiments, in S101, a transaction generally refers to a set of data items that are processed together in a certain operation or at a certain time. A data item is a transaction element in a transaction, which may be a commodity, an attribute, an event, etc. In an application scenario, a transaction may be a purchase behavior of a certain customer, which includes all commodities purchased by the customer.

[0046] As an example, assume that the data set to be processed is transaction data from a supermarket transaction database, which includes the following transaction records:

[0047] Transaction 1: {milk, bread, eggs}

[0048] Transaction 2: {Bread and Butter}

[0049] Transaction 3: {milk, cookies}

[0050] Transaction 4: {Milk}

[0051] Each record represents a transaction, and each commodity in the transaction represents a data item.

[0052] In some embodiments, in S102, an item set refers to a collection of several data items. A frequent item set refers to an item set that appears frequently in a data set to be processed, that is, a set whose support is greater than a first support threshold. Support refers to the frequency with which an item set appears in all transactions. For example, if the support threshold is set to 30%, then only item sets that appear in more than 30% of transactions are considered frequent item sets. Confidence is a measure of the conditional probability that a rule is true in all transactions. Among them, the setting of the first support threshold and the first confidence threshold should meet business objectives and requirements. For example, if the business goal is to discover the most common purchasing patterns, the support threshold may be set higher.

[0053] Figure 2A flowchart of a specific implementation method of S102 provided in an embodiment of the present application is shown. In some embodiments, in order to further accurately obtain frequent item sets and association rules, after setting a first support threshold and a first confidence threshold, an association rule mining algorithm, such as an Apriori algorithm based on a Trie tree, can be executed to find all frequent item sets and association rules from the data set to be processed. Figure 2 As shown, the aforementioned S102 may specifically include the following steps: S1021~S1027.

[0054] S1021: Construct a dictionary tree according to multiple data items in the data set to be processed, where the dictionary tree includes multiple nodes, each node corresponding to a data item.

[0055] Among them, the dictionary tree is a tree data structure used to store a set of data items, in which each node represents a data item, and the constructed dictionary tree is used to quickly retrieve data items.

[0056] S1022: Traverse the dictionary tree to generate multiple candidate item sets, each candidate item set including multiple data items.

[0057] Among them, the candidate itemset refers to the itemset that may be a frequent itemset.

[0058] S1023: For each candidate item set, the support of the candidate item set in the data set to be processed is obtained according to the ratio of the number of occurrences of the candidate item set in the data set to be processed to the total number of transactions in the data set to be processed.

[0059] S1024: Determine candidate item sets whose support is greater than a preset first support threshold as frequent item sets.

[0060] S1025: For each frequent item set, generate at least one candidate rule according to all non-empty subset sets of the frequent item set, where a non-empty subset refers to a set consisting of at least two data items in the frequent item set, and each candidate rule includes a condition item and a conclusion item.

[0061] Among them, a non-empty subset refers to a set consisting of at least two data items selected from a frequent item set. The non-empty subset is used to form the condition item and / or conclusion item of the candidate rule. Specifically, the candidate rule is a potential rule generated from the non-empty subset of the frequent item set. Each candidate rule consists of a condition item (A) and a conclusion item (B). For each frequent item set, at least one candidate rule consisting of its non-empty subset is generated. Because each frequent item set can have multiple non-empty subsets, each frequent item set may generate multiple candidate rules.

[0062] S1026: For each candidate rule, the confidence of the candidate rule is obtained according to the ratio of the support degree of the condition item and the conclusion item appearing simultaneously to the support degree of the condition item appearing.

[0063] S1027: Determine the candidate rules whose confidence is greater than a preset first confidence threshold as association rules.

[0064] In the embodiment of the present application, by setting the first support threshold and the first confidence threshold, the quantity and quality of the generated association rules can be effectively controlled, thereby effectively mining meaningful association rules from the data set to be processed, ensuring the relevance and accuracy of the mining results.

[0065] In an implementation scenario, referring to Table 1 below, it is assumed that the database contains 10 transactions, the item set is {a, b, c, d}, and each transaction consists of two parts: an ID and an item set. Next, the threshold parameters are set, the first support threshold is 0.5, the first confidence threshold is 0.7, the upper limit of the mining pattern length is 3, and the method of S102 is used to perform mining to obtain frequent item sets (as shown in Table 2 below) and association rules (as shown in Table 3 below).

[0066] Table 1

[0067]

[0068] Table 2

[0069]

[0070] Table 3

[0071]

[0072] In some embodiments, in S103, sensitive rules refer to rules that are determined from multiple association rules based on specific needs of users and need to be hidden or specially processed. These rules may contain sensitive information, such as personal identity information, business secrets, etc., and their exposure may have an adverse impact on the data owner or related parties. The key to hiding sensitive rules is how to select appropriate transactions from the data set to be processed and perform data cleaning, which can effectively hide sensitive information and minimize the side effects of the operation. The evolutionary multi-objective optimization algorithm is used to solve the transaction screening problem, and the subset of transactions to be cleaned can be determined. The evolutionary multi-objective optimization algorithm is an algorithm that simulates the biological evolution process to find the optimal solution or solution set. It can process multiple target tasks at the same time and find the best balance between these target tasks.

[0073] The multi-objective optimization model is a mathematical model that aims to minimize multiple decision variables and is used to guide the search process of the evolutionary multi-objective optimization algorithm. By solving the multi-objective optimization model through the evolutionary multi-objective optimization algorithm, the optimal balance point between satisfying multiple objectives can be found and multiple candidate data cleaning strategies can be generated. The evolutionary multi-objective optimization algorithm is used to solve the high-dimensional multi-objective optimization problem of the multi-objective optimization model, which can effectively utilize the evolutionary computing paradigm of the multi-objective optimization algorithm. Multiple candidate data cleaning strategies can be generated in one run, which greatly improves the flexibility and efficiency of data processing.

[0074] The change of decision variables reflects the impact of the data cleaning strategy on the data set to be processed, and is the basis for evaluating the pros and cons of the data cleaning strategy. The specific meanings of each decision variable are as follows:

[0075] The reduction in support of frequent itemsets: indicates the extent to which the support of some frequent itemsets in the dataset to be processed is reduced after data cleaning. This decision variable reflects the impact of data cleaning on the original data distribution and association rules in the dataset to be processed. By evaluating the reduction in support of frequent itemsets, the ability of data cleaning strategies to maintain data relevance and integrity can be effectively guaranteed, and data distortion caused by excessive cleaning can be avoided.

[0076] Confidence change of non-sensitive rules: indicates the degree to which the confidence of non-sensitive association rules in the processed data set changes after data cleaning. This decision variable measures the impact of data cleaning on non-sensitive rules. A small confidence change indicates that the cleaning strategy is more effective in protecting non-sensitive information rules, while a large change may mean that the cleaning strategy also causes unnecessary interference to non-sensitive information. Therefore, evaluating the confidence change of non-sensitive rules helps ensure that the data cleaning strategy protects sensitive information while minimizing the impact on non-sensitive information and maintaining the value of the data.

[0077] Number of modified transactions: indicates the number of modified transactions in the dataset to be processed after data cleaning is performed. This decision variable reflects the breadth and intensity of data cleaning. A large number of modified transactions may mean that the data cleaning strategy covers more data, but may increase the risk of data distortion. The number of modified transactions can be used to evaluate the efficiency and coverage of the data cleaning strategy to ensure that while protecting sensitive information, it does not excessively affect the overall size and quality of the dataset to be processed.

[0078] Number of deleted data items: This indicates the number of data items deleted from the dataset after data cleaning. This decision variable measures the strictness of data cleaning and the direct reduction effect on the dataset. A large number of deleted data items may mean a strict cleaning strategy, but it may also lead to loss of data information. By evaluating the number of deleted data items, it helps to remove sensitive information without excessive loss of valuable data.

[0079] In summary, the reduction in the support of frequent itemsets and the change in the confidence of non-sensitive rules directly reflect the dynamic changes in the statistical characteristics of frequent itemsets and association rules during the data cleaning process, and these changes constitute the side effects at the knowledge level. At the same time, the number of modified transactions and the number of deleted data items intuitively reflect the direct impact at the data level, that is, the side effects at the data level. The data processing method proposed in this application is committed to minimizing these two types of side effects caused by data cleaning operations, so as to maintain the stability of the statistical characteristics of frequent itemsets and association rules while effectively hiding or processing sensitive rules, and reduce distortion at the knowledge level; at the same time, by optimizing transaction screening and data items, the scope of changes at the data level is controlled to ensure the integrity and availability of the data.

[0080] In some embodiments, in order to accurately determine the candidate data cleaning strategies and select the optimal data cleaning strategy, before the aforementioned S103, the method further includes:

[0081] The multi-objective optimization model is constructed using the following objective function:

[0082]

[0083] In the formula, Indicates the search for The smallest solution, is a vector, It is used to represent the reduction of support of all frequent itemsets in the set of frequent itemsets. , It is used to indicate the confidence reduction degree of all association rules with reduced confidence in the non-sensitive rule set. , It is used to indicate the confidence increase of all association rules with increased confidence in the non-sensitive rule set and weak rule set. , , , is the data set to be processed, is the target data set, P is the frequent itemset set, is a frequent item set, is the association rule, is a set of non-sensitive rules. is a weak rule set, m is the number of deleted data items, n is the number of modified transactions, and the weak rule set includes association rules based on frequent item sets and a preset second confidence threshold screening, and the second confidence threshold is less than the first confidence threshold.

[0084] Among them, weak rules may not directly reflect strong correlations or frequent patterns in the data, but may hide important information or knowledge, which may change before and after data cleaning. Therefore, by monitoring these changes, we can further understand the impact of data cleaning operations on data characteristics.

[0085] In the embodiment of the present application, the above decision variables together constitute a multi-dimensional index for evaluating the effectiveness of candidate data cleaning strategies. Through the changes in these decision variables, the performance of candidate data cleaning strategies in protecting sensitive information, maintaining data relevance and integrity, and maintaining the size and quality of data sets can be comprehensively evaluated, thereby selecting the data cleaning strategy with the least side effects, that is, the optimal data cleaning strategy.

[0086] The hiding standard of an association rule is that when the statistical characteristics of an association rule change significantly, for example, its support drops below the second support threshold specified by the user, or its confidence drops below the third confidence threshold specified by the user, then it can be considered that this association rule has been effectively hidden. Based on this, in some embodiments, in order to further limit the feasible solution space of the multi-objective optimization model and effectively ensure that the output of the multi-objective optimization model meets specific requirements or restrictions, constraints can be introduced into the multi-objective optimization model. That is, the aforementioned multi-objective optimization model can also include constraints, which can specifically include reducing the support of sensitive rules to below the second support threshold, and / or reducing the confidence of sensitive rules to below the third confidence threshold.

[0087] In an embodiment of the present application, by reducing the support of specific sensitive rules to below the second support threshold, it can be effectively ensured that these association rules are no longer significant in the data set to be processed. By reducing the confidence of specific sensitive rules to below the third confidence threshold, the credibility and influence of these association rules can be effectively reduced. Furthermore, by adding these constraints to the multi-objective optimization model, the range of feasible solutions of the multi-objective optimization model can be effectively narrowed, the amount of calculation can be reduced, and at the same time, it can be ensured that the data cleaning process can achieve the expected sensitive information protection effect.

[0088] In some embodiments, in S104 , a target data cleaning strategy may be selected from a plurality of candidate data cleaning strategies according to the user's expectations and specific preferences for the data quality in the to-be-processed data set.

[0089] Figure 3A flowchart of a specific implementation method of S103 provided in an embodiment of the present application is shown. In some embodiments, in order to evaluate individuals using a multi-objective optimization model, it is ensured that the generated candidate data cleaning strategy reaches the optimal or approximately optimal level in multiple objectives, such as Figure 3 As shown, the aforementioned S103 may specifically include the following steps: S103a~S103j.

[0090] S103a: Randomly generate an initial population, where each individual in the population represents an initial candidate data cleaning strategy.

[0091] In some embodiments, randomly generating an initial population is the first step of the evolutionary multi-objective optimization algorithm to ensure that there is sufficient diversity in the population so that the evolutionary multi-objective optimization algorithm can explore different regions of the solution space. Specifically, the method of randomly generating an initial population is as follows:

[0092] First, the number of individuals in the population is determined based on the multi-objective optimization model and constraints. Second, each possible data cleaning strategy is mapped to a chromosome using integer coding technology. Each chromosome represents an initial candidate data cleaning strategy, and its coding content is directly related to the specific set of database transaction numbers that need to be deleted. For each sensitive rule in the data set to be processed, the i-th coding segment in the chromosome corresponds to the i-th sensitive rule. , whose rule form is , is the conditional item, is the conclusion item. Its integer encoding content specifies the , the ID set of the transactions in the processed data set that needs to be cleaned. To achieve effective hiding, the length of each encoding segment needs to be determined in advance.

[0093] To ensure sensitive rules After cleaning, if the support or confidence in the processed data set drops below the user-specified threshold, the following formula can be used to calculate the minimum number of transactions that need to be modified, that is, to calculate the length of the i-th coding segment in each chromosome: :

[0094] 1

[0095] In the formula, represents the i-th sensitive rule, Indicates sensitive rules The absolute support (frequency of occurrence) in the dataset to be processed, Indicates sensitive rules is the absolute support of the conditional item in the data set D to be processed, |D| is the total number of transactions in the data set D, MST is the relative support threshold (the second support threshold), and MCT is the relative confidence threshold (the second confidence threshold).

[0096] By taking the smaller value of the two and adding 1 (to ensure that at least one transaction is modified), the support requirement and the confidence limit are met, and the shortest effective length of each encoding segment can be obtained.

[0097] Next, in order to further understand the above , will be The acquisition process is explained.

[0098] To hide sensitive rules The goal of data cleaning is to selectively remove rules containing Specific items in the transaction, thereby reducing the sensitivity of the rules After cleaning, the relative support in the to-be-processed data set D' is reduced to below the second support threshold, or the relative confidence is reduced to below the third confidence threshold. Specifically, it can be expressed as:

[0099]

[0100]

[0101] In the formula, Indicates the number of transactions modified, i.e. .

[0102] It is further deduced that in order to make the sensitive rule The relative support of the cleaned dataset D' is reduced to the second support threshold ( ), the minimum number of transactions that need to be modified in the dataset to be processed is:

[0103]

[0104] Make sensitive rules The relative support of the cleaned data set D' is reduced to Below, the minimum number of transactions that need to be modified for the dataset to be processed is:

[0105]

[0106] S103b: Take the initial population as the current population.

[0107] S103c: Perform crossover and mutation operations on individuals in the current population to generate a descendant population.

[0108] S103d: Pair and select individuals in the current population to generate a parent population.

[0109] In some embodiments, crossover, mutation, pairing, and selection are all operations in a genetic algorithm. Crossover is used to simulate the reproduction process of organisms by exchanging some genes of two individuals to generate new individuals. Mutation is used to increase the diversity of the population by randomly changing some parts of individuals. Pairing is used to pair individuals in pairs for crossover operations. Selection is used to select better individuals to enter the next generation based on the fitness of the individuals.

[0110] S103e: Merge the offspring population and the parent population to obtain a mixed population.

[0111] S103f: Evaluate each individual in the mixed population using a multi-objective optimization model to obtain an evaluation result.

[0112] In some embodiments, for each individual in the mixed population, the multi-objective optimization model is used to calculate the performance value of each individual on each objective to form a fitness vector. Furthermore, these fitness values ​​can be normalized to facilitate comparison and trade-off between different objectives.

[0113] S103g: Based on the evaluation results and the environmental selection algorithm, the mixed population is screened to obtain a reference population.

[0114] In some embodiments, the environmental selection algorithm is a step that simulates the natural selection process, which is used to select individuals with higher fitness from a mixed population to generate the next generation of population. The environmental selection algorithm usually performs selection based on the fitness value of the individual. Specifically, the environmental selection algorithm selects excellent individuals from the mixed population as a reference population based on the evaluation results of the multi-objective optimization model, so as to guide the evolutionary direction of the population while retaining the excellent characteristics in the current population, so that it is closer to the Pareto optimal frontier.

[0115] Specifically, the environment selection algorithm may include roulette selection method, random traversal sampling method, tournament selection method, best individual preservation method, and sorting selection method.

[0116] Roulette selection method: used to determine the probability of an individual being selected based on the proportion of its fitness value in the total fitness value. The higher the fitness value, the greater the probability of the individual being selected.

[0117] Random traversal sampling method: Each individual in the population corresponds to an interval on the roulette wheel, and the selected individual is determined by the randomly generated pointer position.

[0118] Tournament selection method: It is used to randomly select a certain number of individuals for comparison, and then select the individuals with the highest fitness to enter the next generation of the population. It can reduce the impact of super individuals on population evolution and increase the diversity of the population.

[0119] Best individual preservation method: used to directly copy the individual with the highest fitness in the current population to the next generation population to ensure that the optimal solution will not be lost during the evolution process. It can accelerate the convergence speed of the algorithm, but it may also cause the algorithm to mature prematurely.

[0120] Sorting selection method: It is used to sort the population according to the fitness value of the individuals, and then assign a selection probability to each individual according to the sorting result. This method can ensure that individuals with lower fitness also have a certain chance of being selected, thereby increasing the diversity of the population.

[0121] S103h: Update the reference population to the current population.

[0122] S103i: Determine whether the preset termination condition is met. If the determination result is yes, execute S103j, if the determination result is no, return to execute S103c.

[0123] In some embodiments, the preset termination condition may include that the algorithm has executed a preset maximum number of iterations, the population has converged (the change in the population for multiple consecutive generations is very small), a predetermined goal has been achieved (a certain fitness value has been achieved), etc.

[0124] S103j: The current population is determined as the target population, and the candidate data cleaning strategy is any individual in the target population.

[0125] In the embodiment of the present application, the population is continuously improved in an iterative manner, and eventually the optimal solution or a set of high-quality candidate solutions of the multi-objective optimization model is found.

[0126] Next, the above method of the embodiment of the present application will be further explained in conjunction with a specific scenario.

[0127] In an implementation scenario, assume that the set of sensitive rules to be hidden contains two rules: and The transaction ID subsets that support the two rules in the processed data set are {1, 3, 5, 7, 8, 9, 11} and {2, 4, 6, 8, 10, 12}. At least three transactions need to be cleaned, hidden At least two transactions need to be cleaned, so the encoding scheme for each chromosome is designed as follows:

[0128] ( 5 7 11 | 6 10 )

[0129] According to the chromosome encoding, after decoding it, the corresponding candidate data cleaning scheme is:

[0130] To hide , select transactions with IDs 5, 7, and 11 in the data set to be processed for cleaning;

[0131] To hide , select transactions with IDs 6 and 10 in the dataset to be processed for cleaning.

[0132] For the crossover operation, a rearrangement-based method is used to process the merged code segments. Assume that there are two parent solutions (5 7 11 | 6 10) and (3 5 9 | 4 8). The first segment and the second segment of the two parent individuals are merged respectively to obtain the following merged code: (3 5 9 11 | 4 6 8 10).

[0133] Next, each merged fragment is rearranged, that is, the order of the elements in the fragment is randomly disrupted, ensuring that the newly generated offspring individuals are both similar to and different from the parent in terms of genetic information. Then, after rearrangement, elements are extracted from each rearranged fragment according to certain rules (such as selecting the first N elements) to form new offspring individuals. For example, the first three IDs after rearrangement are selected for the first fragment, and the first two IDs after rearrangement are selected for the second fragment as the first offspring individual. After that, each coding fragment is rearranged again, and the second offspring individual is obtained accordingly. For example, the offspring individuals obtained after crossover are: (3 5 11 | 4 6) and (5 9 11 | 8 10).

[0134] For the mutation operation, first, define a mutation probability (such as 0.05), which determines the possibility of mutation of each gene position. Then, traverse each coding segment of the current individual. Each coding segment corresponds to a specific sensitive rule. , and contains a set of transaction IDs that are used to hide the cleansing of the rule. Next, for each gene position in each coding segment (i.e., each transaction ID), a random number is generated. If this random number is less than or equal to the mutation probability (e.g., 0.05), the gene position is determined to mutate. For the gene position that is determined to mutate, the following steps are performed to perform the mutation operation: From the sensitive rule In the corresponding candidate transaction ID set, a new transaction ID is randomly selected to replace the old transaction ID on the current gene position. Finally, the mutated individual is put back into the population to replace the original individual or join the population as a new individual, thereby realizing population update.

[0135] Figure 4A flowchart of a specific implementation method of S104 provided in an embodiment of the present application is shown. In some embodiments, the target transaction includes multiple data items, and the multiple data items include at least one target condition item and at least one target conclusion item. In order to achieve the purpose of hiding sensitive information, the impact on the overall value of the data set to be processed is minimized. Figure 4 As shown, the aforementioned S104 may specifically include the following steps: S1041~S1042.

[0136] S1041: For each target conclusion item in the target transaction, calculate the support of the target conclusion item in the data set to be processed.

[0137] S1042: Identify and delete the target conclusion item with the highest support.

[0138] In the embodiment of the present application, conclusion items with high support may contain more sensitive information. By deleting the target conclusion items with high support, the frequency of occurrence of sensitive rules in the data set can be reduced, thereby reducing their significance. In addition, since the target condition item may be combined with other conclusion items to form multiple different rules, the impact on data integrity can be reduced by selectively deleting the target conclusion item instead of the target condition item, and it is effectively ensured that there are still valuable non-sensitive rules in the data set to be processed.

[0139] Based on the data processing method provided in the above embodiment, the present application also provides a specific implementation of a data processing device. Please refer to the following embodiment.

[0140] See first Figure 5 , the data processing device provided in the embodiment of the present application includes the following modules:

[0141] The acquisition module 501 is used to acquire a data set to be processed. The data set to be processed includes multiple transactions, and each transaction includes at least one data item.

[0142] The determination module 502 is used to mine the processing data set using an association rule mining algorithm to obtain frequent item sets and association rules related to the frequent item sets. The frequent item sets refer to item sets whose support is greater than a preset first support threshold. The item sets include multiple data items. The association rules include rules whose confidence based on the frequent item set screening is greater than a preset first confidence threshold.

[0143] The generation module 503 is used to search for multiple transactions corresponding to each sensitive rule using an evolutionary multi-objective optimization algorithm and a pre-built multi-objective optimization model, and generate multiple candidate data cleaning strategies. The sensitive rule is a rule that needs to be hidden determined from multiple association rules. The candidate data cleaning strategy includes a target transaction to be cleaned. The multi-objective optimization model is constructed with the goal of minimizing multiple decision variables. The multiple decision variables are determined based on multiple parameters determined after data cleaning is performed according to the candidate data cleaning strategy. The multiple decision variables include the reduction in support of frequent itemsets, the change in confidence of non-sensitive rules, the number of modified transactions, and the number of deleted data items.

[0144] The deletion module 504 is used to perform the following operations for each target transaction in the target data cleaning strategy to obtain a target data set, where the target data cleaning strategy is any one of multiple candidate data cleaning strategies: deleting at least one data item related to the sensitive rule in the target transaction.

[0145] In the embodiment of the present application, the acquisition module 501 is used to acquire the data set to be processed, and then the determination module 502 is used to mine the data set to be processed to obtain frequent item sets and association rules. Then, the generation module 503 is used to comprehensively consider multiple decision variables related to frequent item sets and association rules through a multi-objective optimization model, and an evolutionary multi-objective optimization algorithm is used to generate multiple candidate data cleaning strategies, which can balance between various goals so that the total loss of all goals is minimized. Through such optimization, the side effects of data cleaning can be minimized while hiding sensitive information. Finally, the deletion module 504 is used to delete at least one data item related to the sensitive rule in the target transaction.

[0146] Specifically: Minimizing the reduction in support of frequent itemsets can ensure that sensitive information is hidden while reducing the impact on other frequent itemsets and maintaining the integrity of the data set. Minimizing the change in confidence of non-sensitive rules can reduce the impact of data cleaning on non-sensitive association rules and effectively avoid non-sensitive association rules becoming unreliable due to data cleaning. Minimizing the number of modified transactions can reduce the workload and potential errors of data cleaning and effectively maintain the authenticity of the data. Minimizing the number of deleted data items can reduce the impact of data cleaning on the structure of the data set to be processed and maintain the stability of the data.

[0147] As an implementation of the present application, in order to accurately determine the candidate data cleaning strategies and select the optimal data cleaning strategy, the above-mentioned device may further include:

[0148] Build a model for building a multi-objective optimization model using the following objective functions:

[0149]

[0150] In the formula, Indicates the search for The smallest solution, It is used to represent the reduction of support of all frequent itemsets in the set of frequent itemsets. , It is used to indicate the confidence reduction degree of all association rules with reduced confidence in the non-sensitive rule set. , It is used to indicate the confidence increase of all association rules with increased confidence in the non-sensitive rule set and weak rule set. , , , is the data set to be processed, is the target data set, P is the frequent item set, is a frequent item, is the association rule, is a set of non-sensitive rules. is a weak rule set, m is the number of frequent items deleted, n is the number of transactions modified, the weak rule set includes rules screened based on the frequent item set and a preset second confidence threshold, and the second confidence threshold is less than the first confidence threshold; the multi-objective optimization model also includes constraints, and the constraints include: reducing the support of sensitive rules to below the second support threshold, and / or reducing the confidence of sensitive rules to below the third confidence threshold.

[0151] As an implementation of the present application, in order to further accurately obtain frequent item sets and association rules, the above-mentioned determination module 502 may specifically include:

[0152] A first construction submodule is used to construct a dictionary tree according to multiple data items in the data set to be processed, where the dictionary tree includes multiple nodes, each node corresponding to a data item;

[0153] The first generation submodule is used to traverse the dictionary tree to generate multiple candidate item sets, each candidate item set includes multiple data items;

[0154] The first determination submodule is used to obtain the support of each candidate item set in the data set to be processed according to the proportion of the number of occurrences of the candidate item set in the data set to be processed to the total number of transactions in the data set to be processed;

[0155] The second determination submodule is used to determine the candidate item sets whose support is greater than a preset first support threshold as frequent item sets;

[0156] The second generating submodule is used to generate at least one candidate rule for each frequent item set according to all non-empty subset sets of the frequent item set, where the non-empty subset refers to a set consisting of at least two data items in the frequent item set, and each candidate rule includes a condition item and a conclusion item;

[0157] The third determination submodule is used to obtain the confidence of each candidate rule according to the ratio of the support of the condition item and the conclusion item appearing at the same time to the support of the condition item appearing;

[0158] The fourth determination submodule is used to determine the candidate rules whose confidence is greater than a preset first confidence threshold as association rules.

[0159] As an implementation of the present application, the target transaction includes multiple data items, and the multiple data items include at least one target condition item and at least one target conclusion item. In order to achieve the purpose of hiding sensitive information while minimizing the impact on the overall value of the data set to be processed, the above-mentioned deletion module 504 may specifically include:

[0160] The first calculation submodule is used to calculate the support of each target conclusion item in the target transaction in the data set to be processed.

[0161] The first deletion submodule is used to identify and delete the target conclusion item with the highest support.

[0162] As an implementation of the present application, in order to evaluate individuals using a multi-objective optimization model and ensure that the generated candidate data cleaning strategy reaches the optimal or approximately optimal level in terms of multiple objectives, the generation module 503 may specifically include:

[0163] The third generation submodule is used to randomly generate an initial population, and each individual in the population represents an initial candidate data cleaning strategy.

[0164] The fifth determination submodule is used to use the initial population as the current population.

[0165] The fourth generation submodule is used to perform crossover and mutation operations on individuals in the current population to generate a descendant population.

[0166] The fifth generation submodule is used to perform pairing and selection operations on individuals in the current population to generate a parent population.

[0167] The sixth determination submodule is used to merge the offspring population and the parent population to obtain a mixed population.

[0168] The seventh determination submodule is used to evaluate each individual in the mixed population using a multi-objective optimization model to obtain an evaluation result.

[0169] The eighth determination submodule is used to screen the mixed population according to the evaluation results and the environment selection algorithm to obtain a reference population.

[0170] The update submodule is used to update the reference population to the current population.

[0171] The judgment submodule is used to judge whether the preset termination condition is met. If the judgment result is yes, the ninth determination submodule is executed, and if the judgment result is no, the fourth generation submodule is executed.

[0172] The ninth determination submodule is used to determine the current population as the target population, and the candidate data cleaning strategy is any individual in the target population.

[0173] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present application is shown.

[0174] The electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0175] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0176] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 602 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory.

[0177] The memory 602 may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory 602 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0178] The processor 601 implements any one of the data processing methods in the above embodiments by reading and executing computer program instructions stored in the memory 602 .

[0179] In one example, the electronic device may further include a communication interface 603 and a bus 610. Figure 6 As shown, the processor 601, the memory 602, and the communication interface 603 are connected via a bus 610 and communicate with each other.

[0180] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0181] Bus 610 includes hardware, software or both, coupling the components of the electronic device to each other. For example, but not limitation, the bus may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industry standard architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnect, an industry standard architecture (ISA) bus, an infinite bandwidth interconnect, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable bus or a combination of two or more of these. Where appropriate, bus 610 may include one or more buses. Although the present application embodiment describes and illustrates a specific bus, the present application considers any suitable bus or interconnect.

[0182] The electronic device can execute the data processing method in the embodiment of the present application, thereby realizing the combination Figure 1 and Figure 5 The data processing method and device described.

[0183] In addition, in combination with the data processing method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the data processing methods in the above embodiment is implemented.

[0184] In addition, in combination with the data processing method in the above embodiment, the embodiment of the present application can provide a computer program product for implementation. The computer program product includes a computer program, and when the computer program is processed and executed, any of the above data processing methods is implemented.

[0185] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0186] The functional blocks shown in the above structural block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0187] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0188] Aspects of the present disclosure are described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0189] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire a data set to be processed, wherein the data set to be processed includes a plurality of transactions, and each transaction includes at least one data item; Using an association rule mining algorithm to mine the data set to be processed, to obtain frequent item sets and association rules related to the frequent item sets, wherein the frequent item sets refer to item sets whose support is greater than a preset first support threshold, the item sets include multiple data items, and the association rules include rules whose confidence is greater than a preset first confidence threshold based on the screening of the frequent item sets; For each sensitive rule, using an evolutionary multi-objective optimization algorithm and a pre-built multi-objective optimization model, multiple transactions corresponding to the sensitive rule are searched to generate multiple candidate data cleaning strategies, wherein the sensitive rule is a rule that needs to be hidden determined from the multiple association rules, the candidate data cleaning strategy includes a target transaction to be cleaned, and the multi-objective optimization model is constructed with the goal of minimizing multiple decision variables, the multiple decision variables are determined based on multiple parameters determined after data cleaning is performed according to the candidate data cleaning strategy, and the multiple decision variables include a reduction in the support of frequent itemsets, a change in the confidence of non-sensitive rules, the number of modified transactions, and the number of deleted data items; For each target transaction in a target data cleaning strategy, the following operations are performed to obtain a target data set, wherein the target data cleaning strategy is any one of the multiple candidate data cleaning strategies: Deleting at least one data item related to the sensitive rule in the target transaction.

2. The data processing method according to claim 1, characterized in that: Before searching for multiple transactions corresponding to each sensitive rule using an evolutionary multi-objective optimization algorithm and a pre-built multi-objective optimization model to generate multiple candidate data cleaning strategies for each sensitive rule, the method further includes: The multi-objective optimization model is constructed using the following objective function: , where Indicates the search for The smallest solution, is a vector, It is used to represent the reduction of support of all frequent itemsets in the set of frequent itemsets. , It is used to indicate the confidence reduction degree of all association rules with reduced confidence in the non-sensitive rule set. , It is used to indicate the confidence increase of all association rules with increased confidence in the non-sensitive rule set and weak rule set. , , , is the data set to be processed, is the target data set, P is the frequent itemset set, is a frequent item set, is the association rule, is a set of non-sensitive rules. is a weak rule set, m is the number of deleted data items, n is the number of modified transactions, and the weak rule set includes association rules based on frequent item sets and a preset second confidence threshold screening, and the second confidence threshold is less than the first confidence threshold.

3. The data processing method according to claim 1, characterized in that: The method of mining the to-be-processed data set using an association rule mining algorithm to obtain frequent item sets and association rules related to the frequent item sets includes: Constructing a dictionary tree according to the plurality of data items in the data set to be processed, wherein the dictionary tree includes a plurality of nodes, each of which corresponds to a data item; Traversing the dictionary tree to generate multiple candidate item sets, each of the candidate item sets including multiple data items; For each candidate item set, the support of the candidate item set in the data set to be processed is obtained according to the ratio of the number of occurrences of the candidate item set in the data set to be processed to the total number of transactions in the data set to be processed; Determine the candidate item sets whose support is greater than a preset first support threshold as frequent item sets; For each frequent item set, generating at least one candidate rule according to all non-empty subset sets of the frequent item set, wherein the non-empty subset refers to a set consisting of at least two data items in the frequent item set, and each candidate rule includes a condition item and a conclusion item; For each candidate rule, the confidence of the candidate rule is obtained according to the ratio of the support degree of the condition item and the conclusion item appearing at the same time to the support degree of the condition item appearing; The candidate rules whose confidence is greater than a preset first confidence threshold are determined as association rules.

4. The data processing method according to claim 1, characterized in that: The target transaction includes a plurality of data items, the plurality of data items include at least one target condition item and at least one target conclusion item, and the deleting at least one data item related to the sensitive rule in the target transaction includes: For each target conclusion item in the target transaction, calculating the support of the target conclusion item in the data set to be processed; Identify and remove the target conclusion item with the highest support.

5. The data processing method according to any one of claims 1 to 4, characterized in that: The multi-objective optimization model also includes constraints, which include: The support of the sensitive rule is reduced to below a second support threshold, and / or the confidence of the sensitive rule is reduced to below a third confidence threshold.

6. The data processing method according to any one of claims 1 to 4, characterized in that: For each sensitive rule, the evolutionary multi-objective optimization algorithm and the pre-built multi-objective optimization model are used to search for multiple transactions corresponding to the sensitive rule, and multiple candidate data cleaning strategies are generated, including: Randomly generate an initial population, each individual in the population represents an initial candidate data cleaning strategy; Taking the initial population as the current population; The following operations are performed cyclically until the preset termination condition is met, and the current population is determined as the target population. The candidate data cleaning strategy is any individual in the target population: Performing crossover and mutation operations on individuals in the current population to generate a descendant population; Performing pairing and selection operations on individuals in the current population to generate a parent population; Merging the offspring population and the parent population to obtain a mixed population; Using the multi-objective optimization model to evaluate each individual in the mixed population, and obtaining an evaluation result; According to the evaluation results and the environmental selection algorithm, the mixed population is screened to obtain a reference population; The reference population is updated to the current population.

7. A data processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a data set to be processed, wherein the data set to be processed includes a plurality of transactions, and each transaction includes at least one data item; A determination module, configured to mine the to-be-processed data set using an association rule mining algorithm to obtain frequent item sets and association rules related to the frequent item sets, wherein the frequent item sets refer to item sets whose support is greater than a preset first support threshold, the item sets include multiple data items, and the association rules include rules whose confidence is greater than a preset first confidence threshold based on the screening of the frequent item sets; A generation module, for searching for multiple transactions corresponding to each sensitive rule using an evolutionary multi-objective optimization algorithm and a pre-built multi-objective optimization model, and generating multiple candidate data cleaning strategies, wherein the sensitive rule is a rule to be hidden determined from the multiple association rules, the candidate data cleaning strategy includes a target transaction to be cleaned, the multi-objective optimization model is constructed with the goal of minimizing multiple decision variables, the multiple decision variables are determined based on multiple parameters determined after data cleaning is performed according to the candidate data cleaning strategy, and the multiple decision variables include a reduction in support of frequent itemsets, a change in confidence of non-sensitive rules, the number of modified transactions, and the number of deleted data items; The deletion module is used to perform the following operations for each target transaction in the target data cleaning strategy to obtain a target data set, wherein the target data cleaning strategy is any one of the multiple candidate data cleaning strategies: deleting at least one data item related to the sensitive rule in the target transaction.

8. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the data processing method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the data processing method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed, implements the data processing method according to any one of claims 1 to 6.

Images