A differential privacy big data processing method based on programmable switch
By generating noise and perturbing data on a programmable switch, combined with CM-sketch and Bloom filters, the high overhead problem of differential privacy protection systems in big data scenarios is solved, CPU throughput and network efficiency are improved, and efficient privacy protection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2022-09-15
- Publication Date
- 2026-05-19
AI Technical Summary
Existing differential privacy protection systems suffer from high CPU cycle overhead, high network bandwidth consumption, and low privacy budget management efficiency in big data scenarios, resulting in reduced throughput and high maintenance costs.
A differential privacy processing method based on programmable switches is adopted. By generating noise in the switch control plane and perturbing the data plane, combined with the CM-sketch algorithm and Bloom filter, data range limitation and privacy budget are automatically configured, reducing network traffic and computational overhead.
It significantly reduces CPU cycle consumption, lowers network traffic overhead, and improves CPU throughput and data processing efficiency through automated privacy auditing, thus solving the high overhead problem of differential privacy protection in big data scenarios.
Smart Images
Figure CN115480917B_ABST
Abstract
Description
Technical Field
[0001] Differential privacy (DP) is a technique in cryptography that ensures that data queries are indistinguishable from the existence of a particular record, thus providing strong privacy protection: no one querying a dataset can glean any personal information from the data. This invention relates to a method for deploying differential privacy systems, and more particularly to a differential privacy big data processing method based on a programmable switch. Background Technology
[0002] In recent years, with the rapid development of the internet and cloud computing, a large number of users have submitted their personal data to cloud computing platforms for analysis by untrusted third parties to obtain useful information and provide better services to users. Since the analyzed data is often personal and sensitive, data privacy may be compromised without effective protection measures. Differential privacy, with its privacy-preserving properties, can add perturbations to datasets to confuse malicious analysts and prevent them from obtaining personal privacy information. In the context of large-scale data processing, differential privacy has been deployed in several well-known frameworks, such as Hadoop and Spark, to build practical systems that allow untrusted third parties to analyze data while providing privacy guarantees.
[0003] With the rapid growth of data volume, the aforementioned systems need to process this high-speed, massive amount of data in real time to meet the requirements of differential privacy; however, this process incurs significant overhead. How to achieve differential privacy protection within the aforementioned framework with low overhead and high efficiency, and build a practical system, has become a challenge in the actual deployment of differential privacy. When implementing differential privacy protection in big data scenarios, existing system implementation schemes suffer from the following three problems:
[0004] 1) CPU cycle overhead: The process of generating noise and adding perturbations to the data consumes CPU cycles, reducing the overall throughput.
[0005] 2) Network bandwidth issues: A large amount of unnecessary traffic is generated when performing sensitivity calculations, leading to congestion and high latency.
[0006] 3) Privacy budget issue: Every time an analyst executes a query, a privacy budget needs to be calculated and deducted. However, the method of privacy calculation is error-prone and inefficient, resulting in low data utilization and high maintenance costs. Summary of the Invention
[0007] To address the shortcomings of existing technologies, this invention discloses a differential privacy big data processing method based on a programmable switch. This method, based on a programmable switch, can accelerate differential privacy computation in big data scenarios, reduce network traffic, and achieve automatic configuration of privacy budgets.
[0008] The specific technical solution adopted in this invention is as follows:
[0009] A differential privacy big data processing method based on a programmable switch includes the following steps:
[0010] Step 1: Collect the key-value pair information for each piece of data, and check whether the key-value pair information for each piece of data is within the predefined range;
[0011] Step 2: Generate a set of randomly distributed noise values in the control plane of the programmable switch and preload them into the stateful memory of the data plane of the programmable switch; then generate a random number and sample noise from the stateful memory indexed by the random number; add the sampled noise to the data obtained in Step 1 that has not yet been noise-added, obtain data that satisfies differential privacy and forward it to the reducer; the reducer replaces the data that is not within the predefined range with the average value of the predefined range; after aggregating all the data, the result is output.
[0012] Furthermore, step one includes the following sub-steps:
[0013] (1.1) Deploy the CM-sketch algorithm on the data plane of the programmable switch to collect all key-value pair information of each data extracted based on the mapper;
[0014] (1.2) Check each key-value pair of collected data one by one to see if it is within the predefined range. If it is within the predefined range, continue to step two. If it is outside the predefined range, check if the data is appearing for the first time based on the hash table or the exact table. If it is appearing for the first time, the switch will collect the keys of the data that are outside the predefined range into the hash table or the exact table and continue to step two. Otherwise, discard the data.
[0015] Further, step (1.2) specifically includes:
[0016] Create a hash table and an exact table in the programmable switch data plane. Check the key-value pair information of each collected data one by one to see if it is within the predefined range. If it is within the predefined range, continue to step two. If it is outside the predefined range, check whether the data is appearing for the first time based on the hash table or the exact table. If it is appearing for the first time, the hash table records the key in O(1) time complexity. If a hash collision occurs during storage, the key is reported to the programmable switch control plane. The programmable switch control plane inserts these conflicting keys into the exact table and continues to step two.
[0017] Furthermore, step two includes the following sub-steps:
[0018] (2.1) Generate a set of randomly distributed noise values in the control plane of the programmable switch;
[0019] (2.2) Preload a set of randomly distributed noise values into the stateful memory of the programmable switch’s data plane;
[0020] (2.3) Based on the key of the data and the data table, query whether each piece of data has been disturbed by noise. If the key exists in the data table, that is, the data has been disturbed, then the data is forwarded to the reducer. If the key does not exist in the data table, that is, the data has not been disturbed, then generate a random number, sample noise from the stateful memory indexed by the random number, add the sampled noise to the value of the data, and then collect the keys of the data into the data table and forward the data to the reducer. The data table contains the keys of all disturbed data.
[0021] Furthermore, in step (2.1), the set of randomly distributed noise values are noise values that conform to a Laplace distribution.
[0022] Furthermore, in step (2.2), the stateful memory is implemented using a register array, wherein each noise value generated by left shift is scaled to an integer and then preloaded into the register.
[0023] Further, in step (2.3), when perturbing the data, the key-value pairs of the data output by the mapper are shifted by the same proportion, and the sampled shifted noise value is added to the value of the shifted data. The keys of the data with added noise are collected into the data table and then forwarded to the reducer. During forwarding, the shift proportion is encapsulated in the packet header, and the header is also forwarded to the reducer. The reducer shifts the scaled and noise-added data to the right to restore the scaling proportion. The reducer replaces the data that is not within the predefined range with the average value of the predefined range. After all the data is aggregated, the result is output.
[0024] Furthermore, it also includes using Bloom filters to record the dependencies between output key values and mappers, and calculating the privacy budget by querying the dependencies of all output keys.
[0025] Furthermore, a Bloom filter is used to record the dependencies between output keys and mappers. The privacy budget is calculated by querying the dependencies of all output keys.
[0026] a. Divide the Bloom filter into multiple partitions corresponding to each mapper, and record the keys output by each mapper, i.e. the dependency relationship between the keys and the mapper;
[0027] b. Use a Bloom filter to count the number of keys output by the corresponding mapper for each partition;
[0028] c. Calculate the privacy budget based on the maximum number of keys output by the corresponding mapper for each partition.
[0029] Deploying Bloom filters on programmable switches can accelerate the process of calculating the privacy budget that needs to be deducted.
[0030] The beneficial results of this invention are:
[0031] (1) The differential privacy computation is accelerated by using a line-rate data packet processing ASIC pipeline, significantly reducing CPU cycle consumption; (2) Network traffic overhead is reduced by aggregating network traffic and removing unnecessary operations; (3) Privacy auditing is automated through a programmable switch. This invention deploys a differential privacy system within the network, accelerates differential privacy computation, increases CPU throughput, reduces network traffic and privacy auditing overhead, and solves the problem of excessive overhead in implementing differential privacy protection in big data scenarios. Attached Figure Description
[0032] Figure 1 This is a schematic diagram of the architecture of the present invention;
[0033] Figure 2 A schematic diagram of the work process for defining the scope;
[0034] Figure 3 This is a schematic diagram illustrating the data perturbation process.
[0035] Figure 4 This is a diagram illustrating the workflow of a privacy audit. Detailed Implementation
[0036] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. The purpose and effects of the present invention will become clearer. It should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.
[0037] This invention discloses a differential privacy big data processing method based on a programmable switch. The method is implemented using a programmable switch, and its system architecture is as follows: Figure 1 As shown, the method mainly includes a mapper (for obtaining key-value pairs of each data item), a programmable switch (for restricting the data within a predefined range and perturbing the data based on the key-value pairs obtained by the mapper), and a reducer (for outputting the final data that satisfies differential privacy). The method of this invention mainly includes the following steps:
[0038] Step 1: Range Determination: Limit the system output to a predefined range, such as... Figure 2 As shown, the specific steps are as follows:
[0039] (1.1) Execute the primitive sketch_val, deploy the CM-sketch algorithm on the data plane of the programmable switch, use CM-sketch to record all key-value pair information based on each data extracted by the mapper, and go to step (1.2).
[0040] (1.2) The primitive `pool_kv` is executed to collect keys outside the predefined range into a hash table. If a hash collision occurs, the key is reported to the switch control system (the control plane of the programmable switch) and stored in an exact table. Specifically, a hash table and an exact table are created in the programmable switch data plane to record keys outside the predefined range. The hash table records keys quickly in O(1) time complexity. If a hash collision occurs during storage, the key is reported to the switch operating system (i.e., the switch control plane), and the switch operating system inserts the information of these conflicting keys into the exact table.
[0041] (1.3) Execute the primitive check_ran to check if the key value is within the predefined range. If it is within the predefined range, proceed to step two. For key value information that is out of range, check if the data is appearing for the first time based on the hash table or exact table collected in step (1.2). If it is appearing for the first time, store and forward the key value information based on step (1.2) and proceed to step two. Otherwise, discard the data packet.
[0042] Step 2, Data Perturbation: Based on the calculated sensitivity and other parameters, noise is randomly generated and added to the output privacy data to generate the final output value, such as... Figure 3 As shown, the specific steps are as follows:
[0043] (2.1) Execute the primitive `gen_noise` to generate the required random noise distribution based on pre-defined parameters, including the sensitivity value, output range, and specific distribution model required for the noise distribution. Proceed to step (2.2). As one implementation scheme, noise can be generated using the Laplace algorithm, as shown in the following formula:
[0044]
[0045] Parameter description:
[0046] ib is a noise parameter. Where s is the sensitivity, ε is the privacy budget parameter, and x is the noise value.
[0047] ii. The sensitivity s is determined by a predefined range in step one.
[0048] iii.ε is set by the computing platform.
[0049] (2.2) Execute the primitive pool_noise to pool the noise into the stateful memory of the programmable switch’s data plane, and proceed to step (2.3). Specifically, the stateful memory is implemented as a register array. Since the registers in the programmable data plane only store integer values, for floating-point noise values, each noise value generated in the control plane is first shifted left to scale to an integer and then preloaded into the register.
[0050] (2.3) For each extracted key, execute the primitive query_kv. First, query the data table to see if the data corresponding to the key is disturbed by noise. If it is disturbed, the value is forwarded to the reducer and proceeds to step (2.5); otherwise, the value that is not disturbed will be forwarded to step (2.4) to add noise.
[0051] (2.4) Execute the primitives sam_noise and add_noise to randomly sample noise from the register array based on the random number index and add the noise to the value, then proceed to step (2.5). For the noise sampled from the register, when perturbing the data, shift the key-value pairs of the data output by the mapper by the same proportion, and add the sampled shifted noise value to the value of the shifted data by the same proportion, then proceed to step (2.5).
[0052] (2.5) The primitive pool_kv is executed to pool the keys of the perturbed data into the data table. Then it is forwarded to the reducer; during forwarding, the shift ratio is encapsulated in the packet header, and the header is also forwarded to the reducer. The reducer shifts the scaled and noisy data to the right to restore the scaling ratio. For data whose keys are stored in a hash table or exact table that are outside the predefined range, the reducer also executes the primitive repl_val to replace the key-value pairs that are outside the output range with the average value of the predefined range as the final output, so that all data are within the predefined range.
[0053] Furthermore, this invention also includes privacy auditing: calculating and deducting a privacy budget based on the number of mapper output keys, with the following specific steps:
[0054] a. Execute the primitive update_bf to divide the Bloom filter into multiple partitions corresponding to each mapper, record the keys output by each mapper, i.e. the dependency relationship between the key and the mapper, and proceed to step (b).
[0055] b. Execute the primitive get_priv_cost, query the Bloom filter, obtain the key information and the dependency relationship with the mapper, count the number of keys output by each partition, and proceed to step (c).
[0056] c. Execute the primitive `acct_bdg` to calculate the privacy budget based on the maximum number of keys output by each partition obtained in step (b), and then proceed to step (d). The method for calculating the privacy budget to be deducted is as follows:
[0057] P B =ε×N
[0058] Among them, P B ε represents the privacy budget to be deducted, ε is the privacy budget parameter set by the computing platform, and N is the maximum value among the number of keys output by each partition.
[0059] d. Execute the primitive ded_bdg to deduct the privacy budget calculated in step (c).
[0060] This invention is based on the concept of hash mapping and uses a Bloom filter to record the key-value information output by each mapper. The Bloom filter ensures that the query results only contain false positives, i.e., only false alarms and no false negatives. Therefore, the privacy budget deducted will not be too small, thus protecting user privacy. To bear the huge computational overhead of performing hash mapping on each key, this invention deploys the Bloom filter on a programmable switch and performs corresponding hardware optimizations to accelerate the process of recording key-value pairs and mapper dependencies.
[0061] It will be understood by those skilled in the art that the above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.
Claims
1. A differential privacy big data processing method based on a programmable switch, characterized in that, Includes the following steps: Step 1: Collect the key-value pair information for each piece of data, and check whether the key-value pair information for each piece of data is within the predefined range; Step 2: Generate a set of randomly distributed noise values in the control plane of the programmable switch and preload them into the stateful memory of the data plane of the programmable switch; then generate a random number and sample noise from the stateful memory indexed by the random number; add the sampled noise to the data obtained in Step 1 that has not yet been noise-added, obtain data that satisfies differential privacy and forward it to the reducer; the reducer replaces the data that is not within the predefined range with the average value of the predefined range; after aggregating all the data, the result is output.
2. The method according to claim 1, characterized in that, Step one includes the following sub-steps: (1.1) Deploy the CM-sketch algorithm on the data plane of the programmable switch to collect all key-value pair information of each data extracted based on the mapper; (1.2) Check each key-value pair of collected data one by one to see if it is within the predefined range. If it is within the predefined range, continue to step two. If it is outside the predefined range, check if the data is appearing for the first time based on the hash table or the exact table. If it is appearing for the first time, the switch will collect the keys of the data that are outside the predefined range into the hash table or the exact table and continue to step two. Otherwise, discard the data.
3. The method according to claim 2, characterized in that, The specific steps (1.2) are as follows: Create a hash table and an exact table in the programmable switch data plane. Check the key-value pair information of each collected data one by one to see if it is within the predefined range. If it is within the predefined range, continue to step two. If it is outside the predefined range, check whether the data is appearing for the first time based on the hash table or the exact table. If it is appearing for the first time, the hash table records the key in O(1) time complexity. If a hash collision occurs during storage, the key is reported to the programmable switch control plane. The programmable switch control plane inserts these conflicting keys into the exact table and continues to step two.
4. The method according to claim 1, characterized in that, Step two includes the following sub-steps: (2.1) Generate a set of randomly distributed noise values in the control plane of the programmable switch; (2.2) Preload a set of randomly distributed noise values into the stateful memory of the programmable switch’s data plane; (2.3) Based on the key of the data and the data table, query whether each piece of data has been disturbed by noise. If the key exists in the data table, that is, the data has been disturbed, then the data is forwarded to the reducer. If the key does not exist in the data table, that is, the data has not been disturbed, then generate a random number, sample noise from the stateful memory indexed by the random number, add the sampled noise to the value of the data, and then collect the keys of the data into the data table and forward the data to the reducer. The data table contains the keys of all disturbed data.
5. The method according to claim 4, characterized in that, In step (2.1), the set of randomly distributed noise values are noise values that conform to a Laplace distribution.
6. The method according to claim 4, characterized in that, In step (2.2), the stateful memory is implemented using a register array, wherein each noise value generated by left shift is scaled to an integer and then preloaded into the register.
7. The method according to claim 4, characterized in that, In step (2.3), when perturbing the data, the key-value pairs of the data output by the mapper are shifted by the same proportion, and the sampled shifted noise value is added to the value of the shifted data. The keys of the data with added noise are collected into the data table and then forwarded to the reducer. During forwarding, the shift proportion is encapsulated in the packet header, and the header is also forwarded to the reducer. The reducer shifts the scaled and noisy data to the right to restore the scaling proportion. The reducer replaces the data that is not within the predefined range with the average value of the predefined range. After all the data is aggregated, the result is output.
8. The method according to claim 1, characterized in that, It also includes using Bloom filters to record the dependencies between output key values and mappers, and calculating the privacy budget by querying the dependencies of all output keys.
9. The method according to claim 8, characterized in that, A Bloom filter is used to record the dependencies between output keys and mappers. The privacy budget is calculated by querying the dependencies of all output keys. a. Divide the Bloom filter into multiple partitions corresponding to each mapper, and record the keys output by each mapper, i.e. the dependency relationship between the keys and the mapper; b. Use a Bloom filter to count the number of keys output by the corresponding mapper for each partition; c. Calculate the privacy budget based on the maximum number of keys output by the corresponding mapper for each partition.