A MapReduce-based differential privacy protection method for image wavelet band features

Abstract

This paper proposes a MapReduce-based method for differential privacy protection of image wavelet band features: Step 1: Extract image pixel data and metadata such as local path and resolution; Step 2: Upload the extracted image information to HDFS; Step 3: Launch the Mapper program to perform differential privacy protection on the image; Step 4: Launch the Reducer program to aggregate the privacy-protected image data; Step 5: Output the differentially private image data. This method can provide different levels of differential privacy protection for different privacy requirements. It performs multi-level wavelet decomposition on the image and measures the importance of low-frequency wavelet subbands by calculating image energy. Based on the differences in importance of each wavelet subband, a different privacy budget is allocated to each wavelet subband. By performing differential privacy processing on images on a big data platform, this method provides an efficient solution for privacy protection of image big data.

Description

Technical Field

[0001] The present invention relates to a MapReduce-based image wavelet band feature differential privacy protection method, belonging to the technical field of network security. Background Art

[0002] The advent of the big data era has expanded the application scenarios of images, but it has also introduced new security issues that cannot be ignored. Common image protection methods include data encryption and data perturbation. Traditional data encryption methods have many limitations in the big data era due to their complex computational overhead and low data utility. With the development of deep learning models, traditional image data perturbation techniques such as pixelation and box blurring have become inadequate for protecting image privacy. Preventing the leakage of sensitive information in images is a major challenge.

[0003] In 2006, Dwork proposed differential privacy (DP), which effectively addresses the key issues of traditional data perturbation-based privacy-preserving models: their reliance on the attacker's background knowledge and the lack of mathematical proof of privacy protection. DP assumes that the attacker possesses the most robust background knowledge and provides rigorous proof of privacy protection. It is currently one of the most advanced privacy-preserving technologies and has been widely used in fields such as machine learning and recommendation systems.

[0004] Image differential privacy is a field of differential privacy research that has emerged in recent years. Existing research results are primarily categorized into spatial domain processing-based image differential privacy protection methods and transform domain processing-based image differential privacy protection methods, depending on the image processing techniques employed. In 2018, Fan et al. combined spatial pixelation with differential privacy to propose the first image differential privacy protection method. In the same year, Zhang et al. proposed an image differential privacy protection method using matrix low-rank decomposition to protect the differential privacy of matrix eigenvalues. Subsequently, a series of spatial domain processing techniques, such as singular value decomposition and non-negative decomposition, have been proposed for image differential privacy protection. Transform domain processing is another important image processing method, with representative techniques including Fourier transform and wavelet transform. In 2018, Zhang et al. used discrete Fourier transform for image processing and differentially protected the processed images. In 2020, to avoid the drawback of combining Fourier transform with differential privacy techniques, which often introduces excessive noise, Zhang et al. used discrete cosine transform for image processing, improving the quality of the privacy-preserving processed images.

[0005] Wavelet transform is a widely used image processing technique, widely used in fields such as image compression, image enhancement, image denoising, and image fusion. JPEG2000 is a new-generation image compression standard that uses the 5 / 3 integer wavelet transform to achieve lossless image compression and restoration. It aims to replace the discrete cosine transform-based JPEG standard and is currently widely used in the medical imaging field. The integer wavelet transform decomposes an image into low-frequency subbands and high-frequency subbands in three different directions: horizontal, vertical, and diagonal. The low-frequency subbands retain most of the original image information. In complex image applications, it is often necessary to perform multi-layer wavelet decomposition on the image, and to apply different processing methods to the wavelet subbands at different levels. Because each wavelet subband carries different amounts of information and has different processing requirements in different application scenarios, it also requires different privacy protection requirements. Therefore, it is necessary to develop a differential privacy protection method that can provide different privacy protection strengths based on the importance of each wavelet subband. Currently, no research has combined wavelet band characteristics with differential privacy.

[0006] In the era of big data, data volumes are exploding, data types are becoming increasingly diverse, and data structures are becoming increasingly complex. The amount and complexity of computation required to process data are rapidly increasing. The data processing capabilities of individual computers are becoming severely insufficient, necessitating the search for more effective data-intensive parallel computing methods. MapReduce is recognized by industry and academia as the most effective and easy-to-use distributed parallel processing technology for massive amounts of data. As a core component of the Hadoop big data platform, it offers advantages such as ease of scalability, ease of programming for distributed computing, and high fault tolerance. Programs written based on MapReduce can run on clusters consisting of hundreds or thousands of standard computers and reliably process large datasets in parallel, significantly improving data processing efficiency.

[0007] Based on the above technologies, in order to ensure the privacy security of image data stored in big data platforms and promote the application of big data technologies, the present invention innovatively combines the MapReduce computing framework of big data platforms with an image differential privacy protection method based on wavelet band features, providing an effective solution for privacy protection of image big data. Summary of the Invention

[0008] The technology of the present invention solves the following problems: In response to the shortcomings of privacy protection technology for image big data on the Hadoop platform, combined with current mainstream image processing technology, a MapReduce-based image wavelet band feature differential privacy protection method is provided, and the technology is technically implemented on the Hadoop platform to solve the privacy leakage problem of image big data and the problem of efficient parallel implementation on the big data platform.

[0009] The technical solution adopted by the present invention is: a MapReduce-based image wavelet band feature differential privacy protection method, which includes the following steps:

[0010] Step 1: Extract image pixel data and metadata such as local path and resolution

[0011] The big data platform supports TextFile files by default, so it is necessary to extract the image information and convert it into text format. For each image D in the image dataset D to be processed for privacy protection stored in the local file system i , read its pixel matrix And flatten it into a one-dimensional array by row, and for each pixel value in the array, use "," as a separator to splice it into a pixel string Get image D i The absolute path on the local file system and the resolution of the image, the image D i The path information and resolution information are concatenated into a metadata string using "\t" as a separator. The extracted image D i Metadata As the key of MapReduce calculation, data As the value, the string is concatenated using the delimiter "\t" and written to a line of a TextFile file. Other image data in the dataset is appended to the same TextFile file in the same manner, with each line representing a single image. This step extracts key information from image files in formats such as JPEG and PNG, storing it in a text format supported by Hadoop for subsequent computational processing as a lightweight image format used in this invention.

[0012] Step 2: Upload image information to Hadoop file system HDFS

[0013] The local Hadoop client calls a command to upload the TextFile file output in step 1 to the Hadoop file system HDFS for storage. Each image can be uniquely identified on HDFS through the metadata field.

[0014] Step 3: Start the Mapper program to perform differential privacy protection on the image

[0015] S3.1 Read image data on HDFS

[0016] Use the default TextInputFormat input class to read a line of data from the TextFile file stored in HDFS. The driver starts the Mapper program and splits the read line of data according to the predefined delimiter "\t" to obtain the metadata of the image to be processed. and pixel strings Convert pixel string Re-expand into a pixel matrix Composed key-value pairs

[0017] S3.2 Multi-level wavelet decomposition of the image

[0018] The Mapper program calls the 5 / 3 integer wavelet transform tool class to transform the pixel matrix of the image Perform multi-level wavelet decomposition to obtain wavelet sub-bands of the original image at different scales and directions. Generally, 2 to 4 levels of wavelet decomposition can meet the image processing requirements of common application scenarios. This invention uses 3-level wavelet decomposition for illustration.

[0019] S3.3 Allocate privacy budget to each wavelet sub-band

[0020] This step is the core innovation of this invention. Given the total differential privacy budget ε for each image, the privacy budget ε for each wavelet sub-band j is j , using the distribution method ε j =ω p ε p +ω u ε u . Where ε p The privacy budget share allocated to consider the privacy protection requirements of the wavelet band (hereinafter referred to as “privacy protection budget”), ε u The privacy budget share allocated to consider data utility (hereinafter referred to as “data utility budget”), and ε p ≤ε,ε u ≤ε.ω p 、ω u are ε p , ε u The weight and satisfy ω p +ω u =1.

[0021] About Data Utility Budget ε u From the perspective of data utility, data users usually hope that the privacy-protected data has the highest possible data utility. Therefore, the data owner allocates u When , we always allocate as large a value as possible within the range allowed by the total privacy budget ε, that is, ε u=ε. About the privacy protection budget ε p The importance of different wavelet sub-bands is different. Important sub-bands should have higher privacy protection requirements. Therefore, in ε p Assign the largest privacy budget to the least important wavelet band HH1 within the allowed range To avoid wasting privacy budget, In addition, the energy of the most important low-frequency wavelet subband LL3 accounts for the total energy of all subbands Sum = ∑ j Energy j As a measure of the proportion of in is the energy of the low-frequency subband LL3. The importance of all other wavelet subbands is between LL3 and HH1, and the arithmetic progression is used to calculate the energy of the low-frequency subband LL3. The budget value ε is allocated to each of the remaining wavelet subbands in the interval p .

[0022] S3.4 Setting to add random perturbations

[0023] The differential privacy Laplace mechanism is used to add noise to each band coefficient of each wavelet sub-band, and the noise obeys the parameter Δf j / ε j Laplace distribution. Among them, Δf j is the maximum column norm of the coefficient matrix of wavelet subband j, that is, f is the function of the matrix column norm of the query wavelet subband, j′ is the sibling dataset of wavelet subband j defined according to differential privacy, ε j is the privacy budget allocated to wavelet subband j according to the rules in S3.3.

[0024] S3.5 Restoring the original resolution image after privacy protection

[0025] Perform inverse 5 / 3 integer wavelet transform on each wavelet sub-band after differential privacy perturbation, and finally restore the original size image matrix And flatten it into a one-dimensional array by row, and concatenate each pixel in the array with "," to get the pixel string With metadata Write the MapReduce task context together.

[0026] Step 4: Start the Reducer program to collect privacy-protected image data

[0027] The driver starts the Reducer to pull the privacy-protected image information written into the program context by each Map task. and Reducer aggregates the pulled data and Merge and sort in lexicographic order to get new ordered key-value pairs And rewrite the MapReduce task context to wait for output.

[0028] Step 5: Output differentially private image data

[0029] Customize the ImageTextOutPutFormat output class, inherit the FileOutPutFormat class and override the RecordWriter and write methods to read the privacy-protected image key-value pairs written by the Reducer to the MapReduce task context. The final image text is composed according to the data structure of key+“\t”+value in step 1. Write it back to HDFS for permanent storage.

[0030] The advantages of the present invention compared with the prior art are:

[0031] (1) The differential privacy protection method based on the wavelet band characteristics of images proposed in this paper can, to a certain extent, measure the different privacy protection requirements of wavelet band coefficients of different scales after the image is decomposed by wavelet, and can provide differential privacy protection of different strengths for each sub-band according to its privacy protection requirements.

[0032] (2) The present invention combines differential privacy technology with image processing technology and operates based on the MapReduce parallel computing framework of the Hadoop platform. It can be applied in image big data scenarios, thereby improving the usability and performance of the proposed privacy protection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of the importance of wavelet sub-band coefficients;

[0034] Figure 2 Schematic diagram of the parallel data processing operation of the present invention.

[0035] The symbols in the figure are explained as follows:

[0036] IWT stands for 5 / 3 integer wavelet transform;

[0037] I-IWT stands for inverse 5 / 3 integer wavelet transform;

[0038] M Lap represents the differential privacy Laplace noise mechanism;

[0039] Block1, Block i , Blockn Represent the 1st, i-th, and n-th data blocks respectively;

[0040] Map1, Map i , Map n Represent the 1st, i-th, and n-th Map nodes respectively;

[0041] LL n HL n , LH n 、HH n Represents the low-frequency sub-band, horizontal high-frequency sub-band, vertical high-frequency sub-band, and diagonal high-frequency sub-band of the n-th level wavelet decomposition;

[0042] (key1, value1), (key1, value′1), (key i ,value i )、(key i , value′ i )、(key n ,value n )、(key n , value′ n ) represents a key-value pair in the MapReduce processing process. DETAILED DESCRIPTION

[0043] The present invention proposes a MapReduce-based differential privacy protection method for image wavelet band features, which needs to solve the following three problems: (1) When HDFS does not support any image format, how to design an image data format that is convenient for storage in HDFS and processing in MapReduce; (2) How to design a differential privacy budget allocation method to cope with the different privacy protection requirements of each sub-band of the image after wavelet transformation; (3) How to deploy the image differential privacy protection method on a big data platform.

[0044] The specific implementation method of the present invention is described in three parts below:

[0045] 1.Hadoop image file input and output formats

[0046] Hadoop's distributed file system, HDFS, doesn't support any image file formats. Developers can't process image data on Hadoop as easily as they can on Windows or Linux file systems. Therefore, designing image data formats that are convenient for storage on HDFS and computation on MapReduce, based on the file formats supported by the big data platform Hadoop, is crucial for protecting the privacy of image data and for data mining.

[0047] Hadoop supports two file formats: row-oriented and column-oriented. TextFile is the most common file format in the Hadoop ecosystem, storing data by row, using carriage return and line feed characters to separate rows. TextInputFormat is Hadoop's default file input format implementation class, which reads each record line by line and is the most common input method for MapReduce programs. This method reads each data record as a (key, value) pair, where the key is the starting byte offset of the line within the entire file, and the value is the content of the line, excluding any line terminators.

[0048] To facilitate compatibility with Hadoop's common file formats and MapReduce file input formats, the image data format should be designed according to the TextFile data format and TextInputFormat operating mechanism. The designed image data should also be easily converted into a (key, value) format for MapReduce program calculations in the Map and Reduce stages. Considering the practical operational requirements of image differential privacy protection methods for image data, the present invention designs the image data format in two steps:

[0049] (1) Obtain the pixel matrix of the image. The pixel matrix is the most essential data of the image. To protect differential privacy of image data, it is necessary to calculate various parameters such as the differential privacy global sensitivity based on the image pixel matrix to ultimately achieve random perturbation of the image. The pixel matrix of each image is read locally on the HDFS client, and the two-dimensional matrix is flattened into a one-dimensional array and further converted into a string. This allows the image pixel data to be stored row by row in the TextFile. The MapReduce program then restores the one-dimensional pixel array to matrix form during calculation.

[0050] (2) Obtain the image metadata. MapReduce calculates data in the form of (key, value) key-value pairs. The one-dimensional pixel array obtained in (1) can be used as the value. The corresponding key needs to be designed. In the scenario of image differential privacy protection, the image pixels will change due to random perturbations, but the image metadata will always remain unchanged. Therefore, the image metadata is selected as the key for the image in MapReduce calculation. Since each image has a unique absolute path in the local file system of the operating system, the absolute path of the image in the operating system file system can be set as the key. In addition, if the pixel matrix of the image to be protected is not a square matrix, in order to restore the two-dimensional matrix of the image during the calculation process of the MapReduce program, it is necessary to obtain the image resolution and form the key together with the image path.

[0051] Typically, after a Reduce program completes, it writes data back to HDFS using the default file name and data format specified by TextOutputFormat. This poorly reads the data, necessitating a custom file output format. This invention inherits and rewrites the RecordWriter and write methods of the default MapReduce file output class, TextOutputFormat, and implements the ImageTextInputFormat class to meet actual privacy protection requirements.

[0052] 2. MapReduce computing framework

[0053] MapReduce is a distributed computing framework and the core framework for developing Hadoop-based data analysis applications. Its core function is to integrate user-written business logic code and built-in default components into a complete distributed computing program, which is then scheduled through Yarn and runs concurrently on a Hadoop cluster.

[0054] The entire MapReduce framework data processing is divided into two parts: the Map phase and the Reduce phase, which are two programming interfaces, respectively defining the Mapper class and the Reducer class. Users rewrite the corresponding map method and reduce method in the two classes to implement their own core business logic.

[0055] Map: (k1, v1) → [(k2, v2)]

[0056] The input parameter is the data represented by the key-value pair (k1, v1). The processing process is as follows: a row of data records in a file will be passed to the map method in the form of a key-value pair, where the key is the byte offset of the row relative to the start position of the file, and the value is the data in the row; the map method performs user-defined business logic calculations on the input key-value pair, and usually produces intermediate results in another form, which form a key-value pair [(k2, v2)] and are output to the MapReduce context.

[0057] Reduce: (k2, [v2])→[(k3, v3)]

[0058] The input parameter is the key-value pair (k2, [v2]) output by the map method. [v2] represents a set of values, as the same key k2 often contains multiple different output values v2. The reduce method pulls all v2s corresponding to the same key k2 into a set for calculation. The process is to perform the calculation defined by the Reducer on the intermediate calculated values output by the Map phase, and the final output is a key-value pair of the form [(k3, v3)].

[0059] This paper designs and writes map and reduce methods to convert the image dataset to be protected into a dataset protected by differential privacy. The processing process is as follows: Figure 2 The basic process of the parallel data processing model is as follows:

[0060] (1) Preprocessing stage: The image data file is split according to the default block size of MapReduce, and the same number of Map nodes are allocated for parallel computing;

[0061] (2) Map phase: Each Map node runs the Mapper program for distributed parallel computing, reading the data file row by row. Each row of data read corresponds to an image. A 5 / 3 integer wavelet transform is performed on the pixel matrix of each image. Random noise is added to the perturbation using the differential privacy Laplace mechanism. The differentially private image data is then written into the MapReduce context.

[0062] (3) Reduce phase: Start the Reduce program to aggregate the computation results of multiple Map nodes, pull the (key, value) data of each image after differential privacy processing from the MapReduce context, sort it by key value, and write it back into the MapReduce context. Once all image data is aggregated, the MapReduce task ends.

[0063] The MapReduce framework processing described above includes an optional step called Combine. This step occurs after the Map node finishes processing data and before the Reduce node receives the data block. The MapReduce framework uses the Combiner class to pre-aggregate the key-value pairs calculated by the Map node. Typically, the computational output of a Map node is enormous, potentially many times the original input data. Directly transmitting this output to the Reduce node would incur significant network overhead. Furthermore, the number of Reduce nodes is relatively small, and the large amount of data computed in the Map phase is handled by only a small number of Reduce nodes, potentially reducing cluster computing performance. Therefore, to further optimize the computational process, users can design a Combiner class to optimize transmission and computational efficiency. Although the Combiner class does not have its own programming interface, its computational logic is identical to the Reduce method, and therefore, it also inherits from the Reducer class. In the present invention, since the key and value of image data always correspond one-to-one, there will never be a situation where one key corresponds to multiple values during processing, so implementing a Combiner for pre-aggregation is unnecessary.

[0064] 3. Privacy Budget Allocation Method Based on Wavelet Subband Importance Analysis

[0065] Wavelet decomposition of an image yields four subbands, each containing information about the original image in different directions: the low-frequency subband LL retains the main information of the original image, the high-frequency subband HL retains horizontal details, the high-frequency subband LH retains vertical details, and the high-frequency subband HH retains diagonal details. The low-frequency subband LL often requires multiple wavelet decompositions to meet image processing requirements.

[0066] After the image is decomposed by multi-level wavelet, in addition to the low-frequency sub-bands retaining the main information of the original image, it can be seen that the high-frequency sub-bands gradually become clearer as the number of decomposition layers increases. Therefore, it is obvious that different high-frequency sub-band coefficients have different importance. The importance relationship between sub-bands of different resolutions in each layer and between sub-bands of different resolutions in each direction is as follows: Figure 2 As shown, the direction of the arrow indicates the direction from high to low importance.

[0067] The reasonable allocation of privacy budget has always been a hot issue in the field of differential privacy. Given the total privacy budget, a reasonable privacy budget allocation method can better achieve the trade-off between privacy protection and data utility. Figure 2 As can be seen from the above analysis, in the same direction, the lower the wavelet subband resolution, the higher its importance; within the same decomposition level, the importance of each subband is ranked from high to low: low-frequency subband, horizontal high-frequency subband, vertical high-frequency subband, and diagonal high-frequency subband. Obviously, the difference in the importance of each wavelet coefficient leads to different privacy protection requirements for each coefficient. In addition, from the perspective of privacy protection, the more important the wavelet subband coefficient is, the higher the privacy protection requirement is, and thus a smaller privacy budget should be allocated. Image energy is mainly concentrated in the low-frequency subband after wavelet decomposition, so the distortion of the low-frequency subband coefficient will seriously affect the quality of the entire image. Therefore, from the perspective of data utility, a low-distortion strategy should be adopted for low-frequency wavelet subbands, and a larger privacy budget should be allocated as much as possible. In order to achieve more fine-grained differential privacy protection, the present invention designs a privacy budget allocation method based on wavelet subband importance analysis.

[0068] Given the total differential privacy budget allocated to each image as ε, consider the privacy budget ε allocated to each wavelet subband j j , adopt the privacy budget allocation method ε j =ω p ε p +ω u ε u , according to the parallel combination theorem of differential privacy, we know that ε j ≤ε, and ε p ≤ε,εu ≤ε. Where ε p For privacy protection budget, ε u is the data utility budget, ω p 、ω u are ε p , ε u The weight and ω ] +ω u = 1. In practical applications, ω can be set based on the data owner’s preference for data privacy and data utility. p and ω u The present invention focuses on the privacy protection budget ε p The privacy protection budget allocated to the sub-bands with the lowest and highest importance is first determined. and The privacy protection budgets of other sub-bands are allocated according to an arithmetic progression.

[0069] A MapReduce-based image wavelet band feature differential privacy protection method. The specific process is as follows:

[0070] Step 1: Extract image pixel data and metadata such as local path and resolution

[0071] Read the image dataset D to be processed for privacy protection stored in the local file system. For each image D in the dataset D i , read its pixel matrix And flatten it into a one-dimensional array by row, and for each pixel value in the array, use "," as a separator to splice it into a pixel string Read image D i Get the image resolution from the absolute path in the local file system and convert the image D i The path information and resolution information are concatenated into a metadata string using "\t" as a separator. The extracted image D i Metadata and data Use the delimiter "\t" to concatenate as a whole string Write a line to a TextFile file. Other image data in the dataset is appended to the same TextFile file in the same manner, with each line representing an image. This step extracts key information from image files in formats such as JPEG and PNG, using the lightweight image format used in this invention. This information is then stored in a text format supported by Hadoop for subsequent computational processing.

[0072] The image resolution information is used to convert the pixel string into Restore to pixel matrix The absolute path is used to uniquely identify an image on HDFS.

[0073] Step 2: Upload image information to Hadoop file system HDFS

[0074] The local Hadoop client calls a command to upload the TextFile file output in step 1 to the remote Hadoop file system HDFS for storage. Each image can be uniquely identified in the HDFS file through the metadata field.

[0075] The Hadoop big data platform's parallelization strategy is to implement distributed parallel processing through data slicing, assigning different Map nodes to different data slices and independently running the Mapper program on each Map node. From this perspective, if the data and metadata extracted from each image are written into separate TextFiles, and each TextFile is assigned a Map node as a slice, it seems that parallelization can be better achieved. In practice, treating each image as a slice will lead to the classic "small file" problem in Hadoop. Specifically, when the slice file is too small, due to the small amount of data in each slice, the time required for Map tasks to process the data will be far less than the time required for Map task resource allocation, initialization, and shutdown, which in turn seriously affects MapReduce performance. Therefore, the present invention does not adopt this processing method of writing each image into a separate TextFile and treating it as a separate data slice. Instead, all image information is written into a single TextFile file and sliced using the Hadoop system's default file slicing mechanism, i.e., each 128MB of data is considered a slice, and a Map task is assigned. When the image dataset to be processed is large, the default slicing method can achieve better parallel processing results. If there is not actually a large amount of data to be processed, from the perspective of improving data parallel processing, the present invention proposes two recommended processing methods: (1) When the client preprocessor extracts image information, it controls the number of generated TextFiles and increases the number of slices by increasing the number of TextFile files and reducing the size of each TextFile file; (2) Modify the MapReduce running parameters and change the default 128MB slice to a smaller value, such as 64MB slice.

[0076] Step 3: Start Mapper to perform differential privacy protection on the image

[0077] S3.1 Read image data on HDFS

[0078] Since the present invention has taken Hadoop's data input and processing formats into consideration when designing the Hadoop image data format, data processing can be performed conveniently in this step. Each Map task reads a line of data from the TextFile file stored in HDFS through the MapReduce default TextInputFormat input class, and splits the read line of data according to the pre-defined delimiter "\t" to obtain the metadata of the image to be processed. and pixel strings Re-expand the one-dimensional pixel array into a pixel matrix according to the resolution information in the metadata Composed key-value pairs The metadata The pixel matrix does not change during the subsequent MapReduce processing. It will be input into the differential privacy protection algorithm for processing.

[0079] S3.2 Multi-level wavelet decomposition of the image

[0080] The Mapper program calls the 5 / 3 integer wavelet transform tool class to transform the pixel matrix of the image Perform multi-level wavelet decomposition to obtain wavelet sub-bands of the original image at different scales and directions. The simulation experiment of the present invention uses three-level wavelet decomposition to illustrate the invention.

[0081] S3.3 Allocate privacy budget to each wavelet sub-band

[0082] The privacy budget allocation of differential privacy needs to consider two basic theorems: serial combination and parallel combination. The serial combination theorem means that when repeatedly querying the same data set, the privacy budget consumed by each query is ε i , then the privacy budget consumed in the end is the sum of the budget consumed by each query, that is, ε=Σε i The parallel combination theorem divides the original data set into several independent sub-datasets and queries each sub-dataset separately. The final privacy budget consumed is the largest one among all the query budgets, that is, ε = max(ε1,…,ε n ). Wavelet transform decomposes the original image into low-frequency sub-bands and three high-frequency sub-bands in different directions. Obviously, the privacy budget allocation of wavelet sub-bands should follow the parallel combination theorem. Given the total privacy budget ε of an image, the privacy budget ε that can be allocated to each wavelet sub-band j is j The value range of is (0,ε].

[0083] Another issue that needs to be considered in allocating the privacy budget is how to reasonably allocate budget parameters to better balance data privacy and data availability. The smaller the privacy budget allocation, the higher the privacy protection strength and the lower the data utility; conversely, the larger the privacy budget allocation, the lower the privacy protection strength and the higher the data utility. This paper designs an intuitive privacy budget allocation method, which allocates the total privacy budget ε to each wavelet subband j. j Considered as the privacy budget ε p (ε p ≤ε) and data utility budget ε u (ε u ≤ε consists of two parts, and each part is given a certain weight, that is,

[0084] ε j =ω p ε p +ω u ε u

[0085] where ω p +ω u = 1. By adjusting ω p and ω u The data owner can intuitively adjust the allocation of privacy budget according to his or her own preferences for privacy protection and data utility. For example, when the total privacy budget is 1, the data owner sets ω p =0.8,ω u =0.2, which means that the demand for privacy protection is higher and the demand for data utility is lower, and it also means that ω p ε p The adjustment range is (0,0.8], ω u ε u The adjustment range of is (0,0.2]. Since we usually hope that the data utility can be as high as possible, ε u It can be directly set to the maximum value 1 within its value range, that is, ω u ε u =0.2;ε p The actual value of can be set according to the actual data requirements. Generally, in the process of allocating the privacy budget, we hope to minimize the waste of the privacy budget. Therefore, for relatively unimportant wavelet subbands, ε can be allocated. p =1, while more important sub-bands are allocated a smaller privacy protection budget, for example, 0.5. Under this rule, unimportant wavelet sub-bands receive a total privacy budget of 1, while important sub-bands receive a total privacy budget of 0.6. This achieves a fine-grained privacy budget allocation method that can intuitively assign different privacy budgets to wavelet sub-bands of different importance.

[0086] The next question is how to quantify the importance of different wavelet sub-bands, so as to make ε p Assume ε p The maximum value that can be allocated is The privacy budget allocated to the low-frequency subband LL3 is in is the energy of the low-frequency subband LL3, Sum=∑ j Energy j is the sum of all wavelet sub-band energies. Figure 1 It can be seen that sub-band HH1 has the lowest importance, so it is assigned The importance of other sub-bands is between LL3 and HH1, and they are ranked in the order of By this method, the present invention realizes the privacy protection budget ε p objective distribution.

[0087] S3.4 Setting to add random perturbations

[0088] Common differential privacy perturbation mechanisms include the Laplace mechanism and the exponential mechanism. The Laplace mechanism is oriented towards numerical data, while the exponential mechanism is oriented towards non-numerical data. Therefore, this paper adopts the differential privacy Laplace mechanism to add Laplace noise to each band coefficient of each wavelet sub-band. The noise obeys the parameter Δf j / ε j Laplace distribution. Among them, Δf j is the maximum column norm of the coefficient matrix of wavelet subband j, that is, f is the function of the matrix column norm of the query wavelet subband, j′ is the sibling dataset of wavelet subband j defined according to differential privacy, ε j is the privacy budget allocated to wavelet subband j according to the rules in S3.3.

[0089] S3.5 Restoring the original resolution image after privacy protection

[0090] For each wavelet sub-band after differential privacy perturbation, input the inverse 5 / 3 integer wavelet transform tool class in the order of sub-band resolution from small to large, and finally restore the original size differential privacy protected image matrix And convert the noisy image matrix into a string according to the method in step 1 With metadata New key-value pairs are formed and written into the MapReduce task context. Differential privacy has the excellent property of being immune to post-processing. Therefore, even if the wavelet subbands protected by differential privacy undergo an inverse wavelet transform, the privacy protection strength will not be affected. Therefore, according to the parallel combination theorem of differential privacy, the protection of the entire image satisfies ε differential privacy.

[0091] Step 4: Start the Reducer program to collect privacy-protected image data.

[0092] The driver starts the Reducer to pull the privacy-protected image information written into the program context by each Map task. and Reducer aggregates the pulled data and Merge and sort in lexicographic order to get new ordered key-value pairs And rewrite the MapReduce task context to wait for output.

[0093] Step 5: Output differentially private image data

[0094] The MapReduce task will store the calculation results in the specified HDFS directory with the default file name and file content format, and the data readability is poor. The present invention inherits the FileOutPutFormat class, customizes the ImageTextOutPutFormat output class, and rewrites the RecordWriter and write methods. ImageTextOutPutFormat can read the privacy-protected image key-value pairs written by the Reducer into the MapReduce task context, and convert the key-value pairs into the final image text according to the data structure of key+"\t"+value in step 1. Specify the name of the privacy-protected file and write it back to HDFS in TextFile format for permanent storage.

[0095] The contents not described in detail in the specification of the present invention belong to the prior art known to those skilled in the art.

[0096] The above is only a preferred implementation of the MapReduce-based image wavelet band feature differential privacy protection method of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the MapReduce-based image wavelet band feature differential privacy protection method of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the MapReduce-based image wavelet band feature differential privacy protection method of the present invention.

Claims

1. A MapReduce-based image wavelet band feature differential privacy protection method, characterized by: It contains the following steps: Step 1: Extract the image's pixel data, resolution, and local path, convert the image data format that cannot be processed or stored on the Hadoop platform into a lightweight image format, and write it to a local TextFile file. Step 2: Upload image information to Hadoop file system HDFS; The local Hadoop client calls a command to upload the TextFile file output in step 1 to the remote Hadoop file system HDFS for storage; Step 3: Start the Mapper program to perform differential privacy protection on the image; Includes: S3.1 Read image data on HDFS; S3.2 Perform multi-level wavelet decomposition on the image; S3.3 Allocate privacy budget for each wavelet sub-band; S3.4 Perform differential privacy perturbation on each wavelet sub-band; S3.5 Restore the original resolution of the image after privacy protection Step 4: Start the Reducer program to collect privacy-protected image data; Start the Reducer, which aggregates the pulled data and rewrites it into the MapReduce task context for output. Step 5: Output the differentially private image data and output the image data processed by MapReduce to the specified path according to the specified data structure.

2. The MapReduce-based image wavelet band feature differential privacy protection method according to claim 1, characterized in that: In step 1, the image data format that cannot be processed and stored by the Hadoop platform is converted into a lightweight image format. The specific steps are as follows: Read image D i The pixel matrix is flattened into a one-dimensional array by row, and the pixel values in the array are used as "," as separators to concatenate all pixels in sequence into a pixel string Read image D i Resolution, D i The absolute path in the local file system is concatenated into metadata using the "\t" separator in the format of "absolute path + '\t' + resolution". The image data structure is designed as a text format that is convenient for MapReduce processing: "absolute path + '\t' + resolution + '\t' + pixel string".

3. The MapReduce-based image wavelet band feature differential privacy protection method according to claim 1, characterized in that: In step S3.3, a privacy budget is allocated to the wavelet sub-band, and the privacy budget ε is designed. j The distribution method ε j =ω p ε p +ω u ε u , and ε j Split into privacy budget ε p With privacy budget ε u The weighted sum of , ε p With ε u The value range is (0, ε], ω p +ω u =1; Among them, the budget ε u Budget ε for data utility p Budget for privacy.

4. The MapReduce-based image wavelet band feature differential privacy protection method according to claim 3, characterized in that: Image energy is used to measure the importance of low-frequency wavelet subband LL3, and the privacy protection budget ε p To perform quantitative calculations, the steps are as follows: Calculate the energy of all wavelet sub-bands and get the total energy Sum = ∑ j Energy j , with LL3 band energy The proportion of Sum is used as a measure of the importance of LL3; According to the negative correlation between the privacy budget value and the privacy protection requirement, the privacy budget allocated by LL3 is further quantified as 5. The MapReduce-based image wavelet band feature differential privacy protection method according to claim 3, characterized in that: Design a privacy-preserving budget ε p The specific method is as follows: Allocate the largest privacy budget to the least important wavelet band HH1 Allocate privacy budget for the most important wavelet band LL3 The importance of all other wavelet subbands is between LL3 and HH1, and the arithmetic progression is used to The budget value ε is allocated to each other wavelet sub-band in the interval p .

6. The MapReduce-based image wavelet band feature differential privacy protection method according to claim 1, characterized in that: In step S3.4, the differential privacy Laplace mechanism is used to add Laplace noise to each band coefficient of each wavelet sub-band. The noise obeys the parameter Δf j / ε j Laplace distribution.

7. The MapReduce-based image wavelet band feature differential privacy protection method according to claim 1, characterized in that: Step 5 outputs the differentially private image data, and outputs the image data processed by MapReduce to the specified path according to the specified data structure, specifically: Define the image text output class ImageTextOutPutFormat and read the key-value pairs in the program context Use "\t" as the delimiter for and Stitching to get the final image text And output to HDFS storage.

Images