Two-stage adaptive numerical distribution reconstruction method based on local differential privacy
By employing a two-stage adaptive numerical distribution reconstruction method, combined with EMS and EM algorithms, adaptive bucketing and Gaussian kernel smoothing are constructed, solving the reconstruction challenges of numerical data in the LDP environment and achieving a balance between accurate reconstruction and data utilization under high noise conditions.
Patent Information
- Application Number
- CN202511846147.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-09
- Publication Date
- 2026-02-27
AI Technical Summary
Existing numerical data distribution reconstruction methods are difficult to achieve accurate reconstruction in local differential privacy environments. Uniform bucketing strategies lead to excessive noise or loss of details, and existing algorithms perform poorly in high-noise environments.
A two-stage adaptive numerical distribution reconstruction method is adopted, which combines the LDP perturbation mechanism, the two-stage estimation framework, and adaptive bucketing and spatial smoothing techniques. The adaptive bucketing is constructed by the EMS algorithm and quantile inverse mapping, and the accurate reconstruction is performed by the EM algorithm with Gaussian kernel smoothing.
Accurate reconstruction of numerical distributions was achieved in high-noise environments, balancing privacy protection and data availability, and improving the accuracy and stability of the reconstruction results.
Smart Images
Figure CN121580441A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of data privacy protection and statistical analysis technology, and in particular to a method for accurately reconstructing numerical data distribution by an aggregation server while protecting the local data privacy of end users. Background Technology
[0002] With the advent of the big data era, various terminal devices (such as mobile devices and IoT devices) have generated massive amounts of user data. Collecting and analyzing this data is crucial for improving service quality and conducting scientific research. However, this data often contains sensitive user information, and direct collection poses a serious risk of privacy breaches.
[0003] Local Differential Privacy (LDP), as a strong privacy protection standard, has received widespread attention in recent years. Under the LDP framework, data is perturbed by adding noise before leaving the user's device. The server can only collect the perturbed data, thus making it impossible for attackers to deduce the real information of an individual user even if they compromise the server.
[0004] Currently, LDP frequency estimation techniques for categorized data are relatively mature (such as the stochastic response technique). However, reconstructing the distribution of numerical data still faces significant challenges. Numerical data typically exhibits continuity or a large domain, and directly applying existing LDP mechanisms can lead to substantial noise. Existing numerical distribution reconstruction methods usually employ a bucketing strategy to discretize the continuous domain.
[0005] Traditional numerical distribution reconstruction methods, such as expectation-maximization (EM)-based algorithms, typically employ a fixed-width uniform binning strategy. However, real-world data distributions are often non-uniform, potentially containing both high-density peak regions and low-density long-tail regions. In the high-noise environment of LDP (Local Distribution Process), the uniform binning strategy faces a dilemma: if the bins are too wide, important details of the distribution (such as spikes) will be smoothed out; if the bins are too narrow, the sample size in each bin will be too small, leading to extremely large variance in the estimation results and making them susceptible to noise-induced spurious peaks. While the expectation-maximization algorithm (EMS) with indexed smoothing regularization proposed by Li et al. introduces a smoothing mechanism, indexed smoothing based on a uniform grid struggles to adapt to changes in distribution density, often resulting in over-smoothing or insufficient noise suppression. Summary of the Invention
[0006] The purpose of this invention is to overcome the shortcomings of existing technologies. To address the problems existing in existing technologies, this invention proposes a two-stage adaptive numerical distribution reconstruction method based on local differential privacy. This method combines the LDP perturbation mechanism, a two-stage estimation framework, and adaptive bucketing and spatial smoothing techniques, aiming to achieve accurate reconstruction of the original numerical distribution in high-noise environments while balancing privacy protection and data availability.
[0007] To achieve the above objectives, the technical solution provided by this invention is: a two-stage adaptive numerical distribution reconstruction method based on local differential privacy, comprising the following steps:
[0008] Each end user holds a set of numerical private data. End users' privacy data The perturbation is applied using a local differential privacy perturbation mechanism to generate perturbed data. The perturbed data is then sent to an aggregation server. The aggregation server collects the perturbed data from multiple end users, forming a noisy dataset. Based on this noisy dataset, the aggregation server applies a two-stage expectation-maximization algorithm to estimate the true dataset distribution composed of the original data from each user. The first-stage distribution estimation step of the aggregation server uses an expectation-maximization (EMS) algorithm with indexed smoothing regularization to iteratively estimate an initial smooth probability distribution with low variance from the noisy dataset. The aggregation server is based on the initial smoothed probability distribution. Based on the cumulative distribution characteristics, the quantile inverse mapping method is used to calculate the equally probable bucket boundaries, and an adaptive bucketing strategy is constructed. The second-stage estimation step of the aggregation server is based on the adaptive bucketing strategy and the corresponding transition probability matrix. The EM algorithm with spatial kernel smoothing is applied to iteratively estimate the final accurate probability distribution. The aggregation server outputs the precise probability distribution. This is the result of reconstructing the distribution of the original data.
[0009] Specifically, the adaptive bucketing step includes the following steps:
[0010] Let the boundary of the uniform bucket used in the first stage be... , define the domain Divided into equal intervals The intervals (buckets) are as follows, where the first interval (bucket) is... Each bucket represents a range. , No. Each bucket represents a range. The probability estimation vectors corresponding to all buckets are ,in This indicates that the original data estimated by the EMS algorithm falls within the first... The frequency or probability within each bucket; based on Calculate discrete cumulative probability ,in ;
[0011] Construct the domain Piecewise linear cumulative distribution function To perform continuous CDF interpolation, for any :
[0012]
[0013] Set the target number of buckets for the second stage as follows: Generate a uniformly distributed target cumulative probability sequence ,in That is, the first One target probability quantile;
[0014] For each target cumulative probability The corresponding position of the cumulative distribution function in the data domain is found by inverse mapping of the cumulative distribution function. As the new bucket boundary; specifically, finding the satisfying index The inverse solution of linear interpolation is obtained :
[0015]
[0016] Based on the calculated boundary set definition The adaptive bucket, where the first... Each bucket represents a range. , so that in the distribution The probability mass is equal in each bucket.
[0017] Furthermore, the second-stage distribution estimation step includes the following steps:
[0018] Using the bucket boundaries defined by the adaptive bucketing strategy, the noisy dataset is re-statistically analyzed to generate observation vectors aligned with the adaptive buckets and corresponding transition probability matrices. ;
[0019] Calculate the geometric center coordinates of each bucket in the adaptive bucketing strategy. Define a Gaussian kernel function based on the spatial distance between the bucket centers:
[0020]
[0021] in , The coordinates of the bucket center are To smooth bandwidth;
[0022] Based on preset smoothing bandwidth parameters and kernel function Calculation bucket bucket Normalized smoothing weights The calculation formula is as follows:
[0023]
[0024] Execute the EM algorithm, and in each iteration of the EM algorithm, calculate the increase in the log-likelihood function of the current estimated distribution relative to the previous estimated distribution. The growth amount With the preset smoothing threshold Compare; only when The smoothing step is only performed when the value exceeds the smoothing threshold, in order to quickly fit the approximate shape of the distribution in the early stages of the algorithm; if... If the value is less than or equal to the threshold, the smoothing step is skipped, and only a standard expectation-maximization update is performed to avoid over-smoothing of random noise in the later stages of algorithm convergence; when the spatial kernel smoothing expectation-maximization algorithm terminates, the result of the second-stage distribution estimation is output. .
[0025] When the smooth triggering condition is met, the probability estimate vector obtained from the maximization step (M-Step) is... Execution space smoothing, calculation of the corrected probability vector And use it as the input for the next iteration, where the first... The update formula for each bucket is: When the smooth triggering condition is not met, .
[0026] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0027] 1. Through a two-stage strategy, the first stage uses uniform binning and strong smoothing to quickly obtain the approximate outline of the distribution (guided map), which effectively reduces the variance of the initial estimate.
[0028] 2. The adaptive bucketing strategy based on the guided map can dynamically adjust the bucket width according to the density of the data distribution. It automatically densifies the grid in areas with high probability density to preserve details and automatically thins the grid in areas with low probability density to suppress noise, thus achieving a balance between resolution and stability.
[0029] 3. The second stage adopts spatial kernel smoothing based on physical distance, which is adapted to the characteristics of non-uniform buckets; combined with boundary suppression and threshold control mechanisms, it further improves the ability to restore peak values and the robustness to tail noise, thereby realizing accurate reconstruction of numerical distribution in LDP environment. Attached Figure Description
[0030] Figure 1 This is an overall framework diagram of the method of the present invention.
[0031] Figure 2 This is a flowchart of the two-stage numerical distribution estimation method of the present invention.
[0032] Figure 3 This diagram illustrates the difference between equal-probability binning and equal-distance binning in the method of this invention. Detailed Implementation Plan
[0033] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.
[0034] See Figure 1 , Figure 2 and Figure 3 As shown, this embodiment discloses a two-stage adaptive numerical distribution reconstruction method based on local differential privacy, including the following steps:
[0035] Each end user holds a set of numerical private data. End users' privacy data The perturbation is applied using a local differential privacy perturbation mechanism to generate perturbed data. The perturbed data is then sent to an aggregation server. The aggregation server collects the perturbed data from multiple end users, forming a noisy dataset. Based on this noisy dataset, the aggregation server applies a two-stage expectation-maximization algorithm to estimate the true dataset distribution composed of the original data from each user. The first-stage distribution estimation step of the aggregation server uses an expectation-maximization (EMS) algorithm with indexed smoothing regularization to iteratively estimate an initial smooth probability distribution with low variance from the noisy dataset. The aggregation server is based on the initial smoothed probability distribution. Based on the cumulative distribution characteristics, the quantile inverse mapping method is used to calculate the equally probable bucket boundaries, and an adaptive bucketing strategy is constructed. The second-stage estimation step of the aggregation server is based on the adaptive bucketing strategy and the corresponding transition probability matrix. The EM algorithm with spatial kernel smoothing is applied to iteratively estimate the final accurate probability distribution. The aggregation server outputs the precise probability distribution. This is the result of reconstructing the distribution of the original data.
[0036] Specifically, the adaptive bucketing step includes the following steps:
[0037] Let the boundary of the uniform bucket used in the first stage be... , define the domain Divided into equal intervals The intervals (buckets) are as follows, where the first interval (bucket) is... Each bucket represents a range. , No. Each bucket represents a range. The probability estimation vectors corresponding to all buckets are ,in This indicates that the original data estimated by the EMS algorithm falls within the first... The frequency or probability within each bucket; based on Calculate discrete cumulative probability ,in ;
[0038] Construct the domain Piecewise linear cumulative distribution function To perform continuous CDF interpolation, for any :
[0039]
[0040] Set the target number of buckets for the second stage as follows: Generate a uniformly distributed target cumulative probability sequence ,in That is, the first One target probability quantile;
[0041] For each target cumulative probability The corresponding position of the cumulative distribution function in the data domain is found by inverse mapping of the cumulative distribution function. As the new bucket boundary; specifically, finding the satisfying index The inverse solution of linear interpolation is obtained :
[0042]
[0043] Based on the calculated boundary set definition The adaptive bucket, where the first... Each bucket represents a range. , so that in the distribution The probability mass is equal in each bucket.
[0044] Furthermore, the second-stage distribution estimation step includes the following steps:
[0045] Using the bucket boundaries defined by the adaptive bucketing strategy, the noisy dataset is re-statistically analyzed to generate observation vectors aligned with the adaptive buckets and corresponding transition probability matrices. ;
[0046] Calculate the geometric center coordinates of each bucket in the adaptive bucketing strategy. Define a Gaussian kernel function based on the spatial distance between the bucket centers:
[0047]
[0048] in , The coordinates of the bucket center are To smooth bandwidth;
[0049] Based on preset smoothing bandwidth parameters and kernel function Calculation bucket bucket Normalized smoothing weights The calculation formula is as follows:
[0050]
[0051] Execute the EM algorithm, and in each iteration of the EM algorithm, calculate the increase in the log-likelihood function of the current estimated distribution relative to the previous estimated distribution. The growth amount With the preset smoothing threshold Compare; only when The smoothing step is only performed when the value exceeds the smoothing threshold, in order to quickly fit the approximate shape of the distribution in the early stages of the algorithm; if... If the value is less than or equal to the threshold, the smoothing step is skipped, and only a standard expectation-maximization update is performed to avoid over-smoothing of random noise in the later stages of algorithm convergence; when the spatial kernel smoothing expectation-maximization algorithm terminates, the result of the second-stage distribution estimation is output. .
[0052] When the smooth triggering condition is met, the probability estimate vector obtained from the maximization step (M-Step) is... Execution space smoothing, calculation of the corrected probability vector And use it as the input for the next iteration, where the first... The update formula for each bucket is: When the smooth triggering condition is not met, .
[0053] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.
Claims
1. A two-stage adaptive numerical distribution reconstruction method based on local differential privacy, characterized in that, The method comprises the following steps: Step 1, each end user holds a numerical private data , the end user applies a local differential privacy (LDP) perturbation mechanism to perturb the private data , generates perturbed data , and sends the perturbed data to an aggregation server; Step 2, the aggregation server collects the post-disturbance data from a plurality of end users to form a noise data set; Step 3, the aggregation server applies a two-stage expectation maximization algorithm to estimate the true data set distribution from the raw data of each user according to the noisy data set. The aggregation server applies an expectation maximization with index smoothing regularization (EMS) algorithm in the first stage of distribution estimation to iteratively estimate an initial smoothed probability distribution with low variance characteristics from the noisy data set ; The aggregation server calculates equal-probability bucket boundaries using a quantile inverse mapping method according to cumulative distribution characteristics of the initial smooth probability distribution , and constructs an adaptive bucketing strategy; the second-stage distribution estimation step of the aggregation server is based on the adaptive bucketing strategy and a corresponding transition probability matrix , and an EM algorithm with spatial kernel smoothing is applied to iteratively estimate a final accurate probability distribution ; Step 4, the aggregation server outputs the refined probability distribution as a result of the reconstruction of the original data distribution.
2. The numerical distribution reconstruction method of claim 1, wherein, The adaptive bucketing step specifically comprises the following steps: Let the uniform bucket boundaries used in the first phase be , the domain is divided into intervals (buckets) with the st bucket being the interval , the st bucket being the interval , and the probability estimation vector corresponding to all buckets being , where denotes the frequency or probability of the original data falling into the th bucket estimated by the EMS algorithm; the discrete cumulative probability is calculated according to , where ; Constructing a definition domain on piecewise linear cumulative distribution functions to perform continuous CDF interpolation for any :
3. Set the target bucket number of the second stage as , generate a uniform distribution of target cumulative probability sequence , where , that is, the first target probability quantile; For each target cumulative probability , find its corresponding position on the data domain by inverse mapping of the cumulative distribution function as the new bucket boundary; in particular, find the index that satisfies , get by linear interpolation inverse solution:
4. According to the computed set of boundaries Definition adaptive buckets, where the first bucket is the interval such that the probability mass within each bucket is equal under the distribution .
5. The numerical distribution reconstruction method of claim 1 and 2, characterized in that, The second-stage distribution estimation step comprises the following steps: The second-stage distribution estimation step comprises the following steps: recount the noise data set using the bucket boundaries defined by the adaptive bucketing strategy to generate an observation vector aligned with the adaptive buckets and a corresponding transition probability matrix ; calculating a geometric center coordinate for each bucket of the adaptive bucketing strategy defining a Gaussian kernel function based on spatial distance of the bucket center points:
6. wherein , is the bucket center coordinate, is the smoothing bandwidth; Based on a preset smoothing bandwidth parameter And a kernel function , calculating a bucket A normalized smoothing weight Of the bucket , whose calculation formula is:
7. performing the EM algorithm, in each iteration of the EM algorithm, computing the log-likelihood increase of the current estimated distribution relative to the last estimated distribution , the increase is compared with a pre-set smoothing threshold ; only when is greater than the smoothing threshold, a smoothing step is performed to quickly fit the general shape of the distribution in the early stage of the algorithm; if is less than or equal to the threshold, the smoothing step is skipped and only the standard expectation-maximization update is performed to avoid over-smoothing of random noise in the later stage of the algorithm convergence; the spatial kernel smoothing expectation-maximization algorithm outputs the result of the second stage distribution estimation when it is terminated ; when the smoothing trigger condition is satisfied, the probability estimation vector obtained from the maximization step (M-Step) is spatially smoothed to obtain a revised probability vector which is used as the input of the next iteration, where the update formula of the th bin is ; when the smoothing trigger condition is not satisfied, .