Methods, apparatuses, computing devices, and storage media for data processing

Abstract

This application provides a data processing method, apparatus, computing device, and storage medium, belonging to the field of big data technology. The method includes: determining a first query output corresponding to an input dataset based on a query algorithm; determining the perturbation of the first query output by each data record in a target number of sampled data records in the input dataset to the first query output based on a second query output of unsampled data records in the input dataset and the query algorithm; obtaining the sensitivity corresponding to the input dataset; the unsampled data records and the target number of data records forming the input dataset; adding noise to the first query output based on the sensitivity; and outputting the noisy first query output. Using this application can reduce the computational load of sensitivity calculation and improve the efficiency of data processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, and in particular to a data processing method and device, a computing device, and a storage medium. BACKGROUND

[0002] With the development of network technology and computer technology, various institutions pay more attention to the privacy protection of personal data. However, even if these institutions have strict protection of the security of the big data processing system and the anonymity of the data (the data inquirer cannot directly obtain the original data of the individual), the data inquirer can still steal the personal data by inquiring the big data processing system. For example, the data inquirer knows that patient 1 has entered A hospital, and he also knows that patient 1 is the only patient who has been admitted to A hospital on January 1, 2019. The data inquirer wants to know whether patient 1 has cancer, and then inquires the big data processing system of A hospital about the number of patients who have been admitted to A hospital on January 1, 2019 and have cancer. If the output of the inquiry is 1, the data inquirer determines that patient 1 has cancer. As can be seen, even if the big data processing system of the hospital is secure, and the data of the big data processing system is anonymous, the data inquirer can still obtain some personal data. Therefore, in order to more comprehensively protect the privacy of personal data, the differential privacy algorithm has emerged. The principle of the differential privacy algorithm is that the differential privacy algorithm adds noise to the output of the big data processing system, so that the noise can prevent the output of the big data processing system from leaking personal data, and can reduce the impact on the accuracy of the output to a minimum. The specific process is as follows: a third-party data inquirer submits a query algorithm to the server of the data owner. The server determines the query output corresponding to the query algorithm when the input is the input data set, and then uses a differential privacy module to add noise to the query output according to the sensitivity. Finally, the server returns the query output with the added noise to the data inquirer. The sensitivity refers to the maximum change of the query output after adding or deleting a data record from the input data set, that is, the maximum disturbance of a single data record to the query output. As can be seen, when the sensitivity is large, the noise added to the query output will be large, which will seriously affect the accuracy of the query output. Conversely, when the sensitivity is small, the noise added to the query output is small, which may not be able to protect the privacy of the individual. Therefore, it is crucial to determine the sensitivity.

[0003] In related technologies, the sensitivity is determined by using a brute force analysis algorithm. The specific process is as follows: each time a data record is deleted from the input data set, the maximum disturbance of a single data record in the input data set to the query output is determined, that is, the sensitivity is determined.

[0004] Since the brute force analysis algorithm needs to delete one data record in the input data set each time to determine the maximum disturbance of the data record to the output, the calculation amount of the sensitivity is relatively large when the number of data records in the input data set is large, and thus the data processing efficiency is low. SUMMARY

[0005] The present application provides a data processing method, device, computing equipment and storage medium to improve the efficiency of data processing.

[0006] In a first aspect, the present application provides a data processing method, which comprises: determining a first query output corresponding to an input data set according to a query algorithm; determining a disturbance of each data record in a target number of data records sampled in the input data set to the first query output according to a second query output of a data record not sampled in the input data set and the query algorithm, obtaining a sensitivity corresponding to the input data set, the data record not sampled and the target number of data records forming the input data set; and adding noise to the first query output according to the sensitivity, and outputting the first query output after the noise is added.

[0007] The scheme shown in the present application is that the data processing method is executed by a data processing device. After receiving a data query request, the data processing device can determine an input data set corresponding to the data query request. The data processing device can input the input data set to a query algorithm corresponding to the query request to obtain a first query output corresponding to the input data set. The data processing device inputs a data record not sampled in the input data set to the query algorithm, and the output obtained is a second query output. The data processing device uses the second query output and the query algorithm to determine a disturbance of each data record in a target number of data records sampled in the input data set to the first query output. The data processing device determines a maximum disturbance in all disturbances as a sensitivity corresponding to the input data set. The data processing device uses the sensitivity to add noise (the noise can be Laplace noise) to the first query output to obtain the first query output after the noise is added. The data processing device sends the first query output to a terminal used by a data queryer. In this way, since only the target number of data records are used to accurately determine the sensitivity of the input data set, it is not necessary to determine the disturbance of each data record in the input data set to the query output, so the sensitivity can be quickly determined, and thus the query output can be quickly output for the data queryer, improving the efficiency of data processing. Moreover, since the target number is fixed, the calculation amount of the sensitivity is the same regardless of the size of the data set, and unlike the brute force analysis algorithm, the calculation amount of the sensitivity increases rapidly with the increase of the input data set, so the overhead of the present application for calculating the sensitivity is smaller than that of the brute force analysis algorithm for calculating the sensitivity for a data set with a larger number of data records.

[0008] In a possible implementation, before determining the first query output corresponding to the input data set according to the query algorithm, the method further includes: receiving a data query request, the data query request including the query algorithm; and sampling a target number of data records in the input data set corresponding to the data query request to obtain the target number of data records and data records in the input data set that are not sampled.

[0009] According to the scheme, when a data queryer wants to query data in a data processing apparatus, the data queryer inputs content to be queried and a query algorithm used in a terminal. The terminal generates a data query request, and the data query request includes the query algorithm and the content to be queried, and the content to be queried is used to determine an input data set. The terminal sends the data query request to the data processing apparatus, and the data processing apparatus receives the data query request and parses the query algorithm and the content to be queried in the data query request. Then, the data processing apparatus uses the content to be queried to determine the input data set corresponding to the data query request. The data processing apparatus samples a target number of data records in the input data set, and the target number of data records are sampled data records. Data records in the input data set other than the target number of data records can be referred to as unsampled data records. In this way, the data processing apparatus can determine the sampled data records and the unsampled data records.

[0010] In a possible implementation, the determining the first query output corresponding to the input data set according to the query algorithm includes: determining a second query output corresponding to the unsampled data records and a third query output corresponding to the target number of data records according to the query algorithm; and determining the first query output corresponding to the input data set according to the second query output and the third query output.

[0011] According to the scheme, the data processing apparatus can input the unsampled data records to the query algorithm to obtain the second query output. The data processing apparatus can input the sampled data records (i.e., the target number of data records) to the query algorithm to obtain the third query output. The data processing apparatus inputs the second query output and the third query output to the query algorithm to obtain the first query output. In this way, because the big data operators in the query algorithm have associativity and commutativity, the second query output and the third query output can be used to quickly determine the first query output.

[0012] In a possible implementation, according to the second query output of the data records not sampled in the input data set and the query algorithm, the perturbation of each data record in the target number of data records sampled in the input data set to the first query output is determined, and the sensitivity corresponding to the input data set is obtained, including: according to the query algorithm, the third query output corresponding to the target number of data records is determined; according to the query algorithm, the query output after deleting each data record in the target number of data records is determined; according to the second query output of the data records not sampled, the third query output, and the query output after deleting each data record in the target number of data records, the perturbation of each data record in the target number of data records to the first query output is determined; and the maximum perturbation in the perturbation of the target number of data records to the first query output is determined as the sensitivity corresponding to the input data set.

[0013] The scheme shown in the present application can input the sampled data records (i.e., the target number of data records) into the query algorithm, and the output obtained is the third query output. Then the data processing apparatus deletes one data record in the target number of data records in turn, and determines the query output after deleting each data record. The data processing apparatus inputs the second query output and the query output after deleting each data record in the target number of data records into the query algorithm respectively, and obtains the query output corresponding to the deletion of each data record in the target number of data records, which is hereinafter referred to as the plurality of query outputs corresponding to the target number of data records. The data processing apparatus subtracts the first query output from the plurality of query outputs corresponding to the target number of data records respectively, and obtains the perturbation of the target number of data records to the first query output respectively.

[0014] Then the data processing apparatus determines the maximum perturbation in the perturbation of the target number of data records to the first query output, and determines the maximum perturbation as the sensitivity corresponding to the input data set. In this way, since the sensitivity of the input data set can be accurately determined only by using the target number of data records, it is not necessary to determine the perturbation of each data record in the input data set to the first query output, so the sensitivity can be quickly determined, and then the query output can be quickly output for the data queryer, thereby improving the efficiency of data processing.

[0015] In a possible implementation, according to the sensitivity, the first query output is added with noise, and before outputting the first query output added with noise, the method further includes: dividing the input data set into at least two partitions according to the partitions to which the data records belong; determining the current query output of the at least two partitions according to the query algorithm; and determining that the current query output of each partition in the at least two partitions is different from the historical query output.

[0016] The scheme shown in the application will determine whether the input data set constitutes a differential attack before outputting the first query output after adding noise. If it does not constitute a differential attack, the first query output after adding noise is directly output. In this way, differential attacks can be prevented as much as possible.

[0017] In a possible implementation, the method further includes: for a target partition in the at least two partitions, if the current query output of the target partition is the same as the historical query output of the target partition, deleting at least one data record in the target partition so that the current query output of each partition in the at least two partitions is different from the historical query output; determining a query output of the target partition after deleting the at least one data record according to the query algorithm; determining a fourth query output of the input data set according to the query output of the target partition after deleting the at least one data record; and adding noise to the fourth query output according to the sensitivity, and outputting the fourth query output after adding noise.

[0018] The scheme shown in the application will determine whether the input data set constitutes a differential attack before outputting the first query output after adding noise. If it does not constitute a differential attack, the first query output after adding noise is directly output. In this way, differential attacks can be prevented as much as possible.

[0019] In a second aspect, the application provides a data processing apparatus, which includes one or more modules for implementing the method of the first aspect and possible implementation manners thereof.

[0020] In a third aspect, a data processing computing device is provided, which includes a processor and a memory, wherein the memory stores computer instructions, and the processor executes the computer instructions to implement the method of the first aspect and possible implementation manners thereof.

[0021] In a fourth aspect, a computer readable storage medium is provided, which stores computer instructions, and when the computer instructions in the computer readable storage medium are executed by a computing device, the computing device executes the method of the first aspect and possible implementation manners thereof, or the computing device implements the functions of the apparatus of the second aspect and possible implementation manners thereof.

[0022] In a fifth aspect, a computer program product containing instructions is provided, which, when executed on a computing device, causes the computing device to execute the method of the first aspect and possible implementation manners thereof, or causes the computing device to implement the functions of the apparatus of the second aspect and possible implementation manners thereof. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1is a schematic diagram of the combination of big data operators provided by one example embodiment of the present application;

[0024] Figure 2 is a schematic diagram of the exchange of big data operators provided by one example embodiment of the present application;

[0025] Figure 3 is a structural schematic diagram of a computing device provided by one example embodiment of the present application;

[0026] Figure 4 is a system schematic diagram of a data processing method provided by one example embodiment of the present application;

[0027] Figure 5 is a structural schematic diagram of a data processing apparatus provided by one example embodiment of the present application;

[0028] Figure 6 is a flow schematic diagram of a data processing method provided by one example embodiment of the present application;

[0029] Figure 7 is a schematic diagram of determining query output provided by one example embodiment of the present application;

[0030] Figure 8 is a schematic diagram of differential attack provided by one example embodiment of the present application;

[0031] Figure 9 is a schematic diagram of partition being independently processed provided by one example embodiment of the present application;

[0032] Figure 10 is a flow schematic diagram of a data processing method provided by one example embodiment of the present application;

[0033] Figure 11 is a schematic diagram of determining query output provided by one example embodiment of the present application;

[0034] Figure 12 is a schematic diagram of determining query output provided by one example embodiment of the present application;

[0035] Figure 13 is a structural schematic diagram of a data processing apparatus provided by one example embodiment of the present application;

[0036] Figure 14 is a structural schematic diagram of a data processing apparatus provided by one example embodiment of the present application;

[0037] Figure 15 is a structural schematic diagram of a data processing apparatus provided by one example embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the drawings.

[0039] In order to facilitate the understanding of the embodiments of the present application, the concepts of the terms involved in the embodiments of the present application will be introduced first as follows:

[0040] Big data processing system, a program for parallel computing a large amount of data by using multiple devices, such as Apache Spark, Camdoop, DryadLINQ, Pregel, etc.

[0041] Data record, a row of data information in a data set, which is a unit constituting the data set.

[0042] Big data operator, a function with associativity and commutativity, such as the map operator (map) and the reduce operator (reduce) of Apache Spark.

[0043] Associativity: When the data records in the input data set are divided into multiple non-overlapping sets, the big data operator can first calculate the output of each set, and then take the outputs of all sets as the input of the big data operator, and the output obtained is the output of the input data set inputting the big data operator. For example, as shown in FIG. 1, the big data operator is used for summation, the input data set ({1, 4, 2, 3, 1, 2, 3, 1}) is inputted into the big data operator, and the output is 17. The input data set ({1, 4, 2, 3, 1, 2, 3, 1}) is divided into two non-overlapping sets (i.e. {1, 4, 2, 3} and {1, 2, 3, 1}), {1, 4, 2, 3} is inputted into the big data operator, and the output is 10, {1, 2, 3, 1} is inputted into the big data operator, and the output is 7. 10 and 7 are inputted into the big data operator, and the output is 17. Figure 1

[0044] Commutativity: The big data operator can allow the records of the input data set to be inputted into the big data operator in any order, and the output of the big data operator is the same regardless of the order of the data records of the input data set inputted into the big data operator, which is not affected by the input order of the data records. For example, as shown in FIG. 2, the big data operator is used for summation, the input data set is {1, 4, 2, 3, 1, 2, 3, 1}, and the input data set is inputted into the big data operator in various orders such as {1, 4, 2, 3, 1, 2, 3, 1}, {1, 1, 4, 2, 3, 1, 2, 3}, {3, 1, 1, 4, 2, 3, 1, 2}, …, {4, 2, 3, 1, 2, 3, 1, 1}, etc., and the output is 17. Figure 2

[0045] ​​The mapping operator, one of the commonly used operators of Apache Spark, is a one-to-one function with associativity and commutativity. The one-to-one function refers to a function with only one data record in the input and output. The main application of the mapping operator is to change the value or type of the value of a single data record, such as converting a string to a number.

[0046] The reduction operator, one of the commonly used operators of Apache Spark, is a many-to-one function with associativity and commutativity. The many-to-one function refers to a function with multiple data records in the input and only one value in the output. The main application of the reduction operator is to combine the values of multiple data records, such as summing the values of multiple data sets to obtain an output.

[0047] The query algorithm is composed of big data operators, such as mapping operators and reduction operators. The query algorithm can be any big data algorithm. For example, algorithms for summing and algorithms for subtracting.

[0048] The sensitivity refers to the maximum change in the query output caused by adding or deleting a data record from the input data set. The query output is the output obtained by inputting the input data set into the query algorithm.

[0049] In the field of big data, in order to protect the privacy of personal data, the sensitivity is used to add noise to the query output. The noise is determined based on the sensitivity of the input data set, so it is crucial to quickly and accurately determine the sensitivity. In related technologies, when determining the sensitivity, a brute force analysis algorithm is used to determine the sensitivity. The specific processing is as follows: each time a data record is deleted from the input data set, the maximum disturbance of a single data record in the input data set to the query output is determined, that is, the sensitivity is determined. Since the brute force analysis algorithm needs to delete a data record from the input data set each time to determine the maximum disturbance of the data record to the output, the calculation amount of the sensitivity is relatively large in the case of a large number of data records in the input data set, which further leads to a relatively low efficiency of data processing. Therefore, the present application provides a data processing method to quickly and accurately determine the sensitivity and to efficiently process data.

[0050] Before introducing the data processing method provided by the embodiments of the present application, the system architecture to which the embodiments of the present application are applied is introduced.

[0051] The data processing method can be executed by a data processing device. The data processing device can be a hardware device, such as a server, a terminal computing device, etc., or a software device (such as a set of software programs running on a hardware device).

[0052] The data processing apparatus can be deployed in a cloud environment, which is an entity that provides cloud services to users by utilizing underlying resources in a cloud computing mode. The cloud environment includes a cloud data center and a cloud service platform. The cloud data center includes a large number of underlying resources (including computing resources, storage resources, and network resources) owned by a cloud service provider. The computing resources included in the cloud data center can be a large number of computing devices (for example, servers). The data processing apparatus can be a server for data processing in the cloud data center. The data processing apparatus can also be a virtual machine for data processing created in the cloud data center. The data processing apparatus can also be a software apparatus deployed on a server or a virtual machine in the cloud data center, which is used for data processing. The software apparatus can be distributed and deployed on multiple servers, or distributed and deployed on multiple virtual machines, or distributed and deployed on virtual machines and servers.

[0053] When the data processing apparatus is a software apparatus, the data processing apparatus can be logically divided into multiple parts, each part having a different function (for example, the data processing apparatus can include a determination module, an output module, etc.). Several parts of the data processing apparatus can be respectively deployed on different environments or devices, and the parts of the data processing apparatus deployed on different environments or devices cooperate to achieve the function of data processing. It should be understood that the present application does not limit the division of the parts of the data processing apparatus, nor does it limit the specific deployment of the data processing apparatus in an environment. In actual application, adaptive deployment can be made according to the computing power of each computing device or specific application requirements.

[0054] When the data processing apparatus is a software apparatus, the data processing apparatus can also be separately deployed on one computing device in any environment (cloud environment, terminal computing device, etc.). When the data processing apparatus is a hardware device, the data processing apparatus can be a computing device 300 in any environment. Figure 3 A structural diagram of a computing device 300 is provided, Figure 3 The computing device 300 shown includes a memory 301, a processor 302, a communication interface 303, and a bus 304. The memory 301, the processor 302, and the communication interface 303 are communicatively connected to each other through the bus 304.

[0055] The memory 301 can be a read only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The memory 301 can store computer instructions, when the computer instructions stored in the memory 301 are executed by the processor 302, the processor 302 and the communication interface 303 are used to execute the method of data processing. The memory can also store data, for example, part of the memory 301 is used to store the data required by the method of data processing, and is used to store intermediate data or result data in the process of program execution.

[0056] The processor 302 can adopt a general central processing unit (CPU), an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or any combination thereof. The processor 302 can include one or more chips, and the processor 302 can include an artificial intelligence (AI) accelerator, for example, a neural processing unit (NPU).

[0057] The communication interface 303 uses a transceiver module such as but not limited to a transceiver to realize the communication between the computing device 300 and other devices or communication networks. For example, the data required in the process of data processing can be obtained through the communication interface 303.

[0058] The bus 304 can include a channel for transmitting information between the components of the computing device 300 (for example, the memory 301, the processor 302, the communication interface 303).

[0059] In order to better understand the embodiments of the present application, as Figure 4 shown, the whole process of data query is provided, that is, the terminal of the data queryer sends a query request to the device for data processing, the query request includes a query algorithm, the device for data processing can calculate the sensitivity of the query output and the input data set, add noise to the query output based on the sensitivity, and return the query output with noise to the terminal of the data queryer.

[0060] The device for data processing can be logically divided into three modules when executing the method of data processing, which are respectively a sampling and partitioning module, a module for determining the sensitivity and partitioning the query output (which can be referred to as a determination module) and a module for processing the query output (which can be referred to as an output module). The determination module can be composed of big data operators. For example, as Figure 5As shown, the determining module can include 3 big data operators, etc.

[0061] The sampling and partitioning module is configured to divide the input data set into partitions and randomly sample a target number of data records in the input data set, so that the input data set is divided into sampled data records and non-sampled data records.

[0062] The determining module is configured to determine the perturbation of each data record in the sampled data records to the query output of the input data set and the query output of the partition.

[0063] The output module is configured to add noise to the query output of the input data set, etc.

[0064] Of course, the data processing apparatus can also include a receiving module configured to receive a data query request, etc.

[0065] It should be noted that the query algorithm includes big data operators, and the determining module is also composed of big data operators. When the data processing method is executed, the big data operators in the determining module collectively implement the query algorithm.

[0066] The data processing method provided in the embodiments of the present application will be described below. Figure 6 The data processing method provided in the embodiments of the present application will be described below. Figure 6 As shown, the processing flow of the method is as follows:

[0067] In step 601, the data processing apparatus determines a first query output corresponding to an input data set according to a query algorithm.

[0068] In this embodiment, after receiving a data query request, the data processing apparatus can determine an input data set corresponding to the data query request. The data processing apparatus can input the input data set into the query algorithm corresponding to the query request to obtain the first query output corresponding to the input data set.

[0069] For example, the query algorithm corresponding to the query request is an algorithm for summing, the input data set is {1, 4, 2, 3, 1, 2, 3, 1}, and the query output is 17.

[0070] In step 602, the data processing apparatus determines the perturbation of each data record in a target number of sampled data records in the input data set to the first query output according to the second query output of the non-sampled data records in the input data set and the query algorithm, and obtains the sensitivity corresponding to the input data set.

[0071] The target number is a large number, such as 1000. The target number of data records is composed of the data records sampled (or selected) from the input data set. The data records not sampled are the data records in the input data set other than the target number of data records.

[0072] In this embodiment, the data processing apparatus can input the data records not sampled in the input data set to the query algorithm to obtain the second query output. The data processing apparatus can use the second query output and the query algorithm to determine the perturbation of each of the target number of data records sampled in the input data set to the first query output. The data processing apparatus determines the maximum perturbation among all the perturbations as the sensitivity corresponding to the input data set.

[0073] In step 603, the data processing apparatus adds noise to the first query output according to the sensitivity, and outputs the first query output after adding noise.

[0074] In this embodiment, the data processing apparatus can use the sensitivity to add noise (which can be Laplace noise) to the first query output to obtain the first query output after adding noise. The data processing apparatus sends the first query output to the terminal used by the data queryer. Specifically, when the sensitivity is low, the noise added to the first query output is small, and when the sensitivity is high, the noise added to the first query output is large.

[0075] In this way, since the sensitivity of the input data set can be accurately determined using only the target number of data records, it is not necessary to determine the perturbation of each data record in the input data set to the query output, so the sensitivity can be quickly determined, and then the query output can be quickly output to the data queryer. In addition, since the target number is fixed, the amount of calculation for determining the sensitivity is the same regardless of the size of the data set, and unlike the brute force analysis algorithm, the amount of calculation for determining the sensitivity does not increase rapidly with the increase of the input data set. Therefore, for a larger data set, the overhead of calculating the sensitivity according to the present application is smaller than that of calculating the sensitivity according to the brute force analysis algorithm.

[0076] The reason why the sensitivity of the input data set can be accurately determined is that:

[0077] The perturbation of a single data record to the query output approximately obeys a normal distribution (most data records have small perturbation to the query output, and a small part of data records have large perturbation to the query output). For an input data set obeying a normal distribution, regardless of the size of the input data set, randomly sampling a target number (such as 1000) of samples from the input data set is sufficient to accurately fit the normal distribution obeyed by the input data set. Therefore, for an input data set, the sensitivity estimated by the present application is the same as the actual sensitivity in statistical theory.

[0078] In a possible implementation, before step 601, the following processing is further performed:

[0079] In step 600, the data processing apparatus receives a data query request, the data query request including a query algorithm, randomly sampling a target number of data records in an input data set corresponding to the data query request, obtaining the target number of data records and data records in the input data set that are not sampled.

[0080] In this embodiment, when the data querier wants to query data in the data processing apparatus, the content to be queried and the query algorithm used are input in the terminal. The terminal can generate a data query request including the query algorithm and the content to be queried, and the content to be queried is used to determine the input data set, such as patient information hospitalized from May 10, 2020 to May 30, 2020. Here, the query algorithm can be a specific algorithm execution code or an identifier of the query algorithm.

[0081] The terminal sends the data query request to the data processing apparatus, and the data processing apparatus receives the data query request and parses the query algorithm and the content to be queried in the data query request. Then, the data processing apparatus uses the content to be queried to determine the input data set corresponding to the data query request. The data processing apparatus randomly samples a target number of data records in the input data set, and the target number of data records are the sampled data records. The data records in the input data set other than the target number of data records can be referred to as the unsampled data records. In this way, the data processing apparatus can determine the sampled data records and the unsampled data records. In addition, if the query algorithm in the query request is an execution code, the data processing apparatus directly runs the execution code subsequently. If the query algorithm in the query request is an identifier of the query algorithm, the data processing apparatus uses the identifier to obtain the execution code of the query algorithm, and then directly runs the execution code.

[0082] In addition, in step 600, the terminal of the data querier sends the query request to the data processing apparatus, and the query request further includes an identifier of the data querier. The data processing apparatus can parse the identifier of the data querier in the data query request when receiving the data query request. The data processing apparatus determines whether the identifier is in a list of trusted identifiers (the data queriers in the list of trusted identifiers directly output the results when querying without needing noise processing), and determines the query output of the input data set if the identifier of the data querier is in the list of trusted identifiers, and directly outputs the query output. If the identifier of the data querier is not in the list of trusted identifiers, the noise processing procedure is performed.

[0083] In a possible implementation, the first query output can be determined using commutativity and associativity of big data operators, and the processing of step 601 is as follows:

[0084] The data processing apparatus determines, according to the query algorithm, a second query output corresponding to the data records that are not sampled and a third query output corresponding to the target number of data records, and determines, according to the second query output and the third query output, the first query output corresponding to the input data set.

[0085] In this embodiment, the data processing apparatus can input the data records that are not sampled into the query algorithm, and the obtained output is the second query output. The data processing apparatus can input the sampled data records (i.e., the target number of data records) into the query algorithm, and the obtained output is the third query output. The data processing apparatus inputs the second query output and the third query output into the query algorithm, and obtains the first query output.

[0086] For example, the input data set is {1, 4, 2, 3, 1, 2, 3, 1}, the target number of data records is {1, 4, 2}, the data records that are not sampled are {3, 1, 2, 3, 1}, and the query algorithm is used to sum. The data processing apparatus inputs {1, 4, 2} into the query algorithm, and the output is 7 (i.e., the third query output). The data processing apparatus inputs {3, 1, 2, 3, 1} into the query algorithm, and the output is 10 (i.e., the second query output). The data output apparatus inputs 10 and 7 into the query algorithm, and the output is 17 (i.e., the first query output).

[0087] In a possible implementation, in step 602, the processing of determining the sensitivity corresponding to the input data set is as follows:

[0088] The data processing apparatus determines, according to the query algorithm, a third query output corresponding to the target number of data records. According to the query algorithm, the data processing apparatus determines a query output after deleting each data record in the target number of data records respectively. According to the second query output of the data records that are not sampled, the third query output, and the query output after deleting each data record in the target number of data records respectively, the data processing apparatus determines a perturbation of each data record in the target number of data records to the first query output. The data processing apparatus determines the maximum perturbation in the perturbations of the target number of data records to the first query output as the sensitivity corresponding to the input data set.

[0089] In this embodiment, the data processing apparatus can input the sampled data records (i.e., the target number of data records) into the query algorithm, and the obtained output is the third query output. Then the data processing apparatus deletes one data record in the target number of data records at a time, and determines the query output after deleting each data record.

[0090] The data processing device inputs the second and third query outputs of the unsampled data records into the query algorithm to obtain the first query output. The data processing device then inputs the second query output and the query output after deleting each data record from the target number of data records into the query algorithm, respectively, to obtain the query outputs corresponding to the respective deletions of each data record from the target number of data records; these are hereinafter referred to as the multiple query outputs corresponding to the target number of data records. The data processing device subtracts the first query output from each of the multiple query outputs corresponding to the target number of data records to obtain the perturbations of the first query output by the target number of data records.

[0091] The data processing device then determines the maximum perturbation of the target number of data records on the first query output, and identifies this maximum perturbation as the sensitivity of the input dataset. Thus, since the sensitivity of the input dataset can be accurately determined using only the target number of data records, it is not necessary to determine the perturbation of each data record in the input dataset on the query output. Therefore, the sensitivity can be quickly determined, and the query output can be quickly provided to the data queryer.

[0092] For example, such as Figure 7 As shown, the input dataset is {1,4,2,3,1,2,3,1}, the target number of data records is {1,4,2}, and the unsampled data records are {3,1,2,3,1}. The query algorithm is used for summation. The data processing device inputs {1,4,2} into the query algorithm, and the output is 7 (i.e., the third query output). The data processing device inputs {4,2}, {1,2}, and {1,4} into the query algorithm, and the outputs are 7 (i.e., the third query output), 6, 3, and 5 respectively. The data processing device inputs {3,1,2,3,1} into the query algorithm, and the output is 10 (i.e., the second query output). The data output device inputs 10 and 7 into the query algorithm, and the output is 17 (i.e., the first query output). The perturbation of data record 1 to the first query output is 17-16=1, the perturbation of data record 2 to the first query output is 17-15=2, and the perturbation of data record 4 to the first query output is 17-13=4. Thus, the maximum perturbation is 4, and the sensitivity of the input dataset {1,4,2,3,1,2,3,1} is 4. Figure 7 A slash on a data record indicates that the data record has been deleted.

[0093] In one possible implementation, even Figure 6The estimated sensitivity in the illustrated process deviates from the actual sensitivity. This application's embodiments can detect and prevent differential attacks, providing differential privacy protection for the input dataset as much as possible. First, let's introduce differential attacks: Given two query algorithms Q and Q' (the input datasets corresponding to Q and Q' are D and D', respectively), a "differential attack" is defined as query algorithms Q and Q' having the same input-output mapping, and D and D' differing by only one data record. For example... Figure 8 As shown, the data queryer submits two algorithms with the same input-output mapping (summation in this case) to the data processing device, and requests that data records (4) in the input dataset be filtered out when submitting query algorithm Q'. The input dataset D{1,4,2,3,1,2,3,1} of query algorithm Q and the input dataset D'{1,2,3,1,2,3,1} of query algorithm Q' differ by only one data record (4). The data queryer can determine from the difference in the query outputs of the two query algorithms (the difference between query outputs 17 and 13 is 4) that the data record "4" exists in the input dataset.

[0094] To better prevent differential attacks, the principles of detection and prevention of differential attacks in this application also utilize the commutativity and associativity of big data operators. The commutativity and associativity of big data operators ensure that each data record or subset of the input dataset (i.e., the partitions mentioned later) is processed independently. Therefore, if the query output of one data record or partition changes, the query output of another record or partition will not be affected. Specifically, in the input dataset D, D1 (partition one) and D2 (partition two) are two non-overlapping partitions of D, and D1 and D2 are merged into D (D1∪D2=D). The query output of D1 is O1=Q(D1), and the query output of D2 is O2=Q(D2). If D1 is missing one data record, O1 may change, but O2 will not change.

[0095] For example, such as Figure 9 As shown ( Figure 9D1 is {1, 4, 2, 3}, the last four data records of the input data set form partition two (i.e., D2 is {1, 2, 3, 1}), the query algorithm is an algorithm for summing, if the first data record in D1 is deleted, the query output of D1 decreases from 10 to 9, and the query output of D2 remains unchanged (the query output is still 7), the query output of the input data set decreases from 17 to 16. This is because the commutativity and associativity of the big data operator allow each partition to be processed independently. Therefore, the embodiment of the application determines whether the current query output can constitute a differential attack (i.e., whether the input and output mapping of the current query and the past query of the query algorithm are the same, and the input data set only differs by one record) by comparing the query output of the current query of the partition (i.e., the current query output) and the query output of the past query of the partition (i.e., the historical query output).

[0096] The following provides a flow of how to prevent differential attacks in a data processing process, as shown in Figure 10

[0097] Step 1001, the data processing apparatus divides the input data set into at least two partitions according to the partition to which each data record belongs.

[0098] In the embodiment, the data processing apparatus has divided the partitions when storing the data records. Alternatively, the device storing the input data set has divided the partitions when storing the data records. For each data record, the partition is divided when the data record is stored, that is, the identifier of all the partitions is stored for each data record. Specifically, when the partitions are divided, the number of data records included in each partition is basically the same.

[0099] The data processing apparatus can divide the input data set into at least two partitions according to the partition to which each data record in the input data set belongs.

[0100] In a possible implementation manner, in order to make the processing resources occupied by the processing of preventing differential attacks relatively small, the input data set is divided into two partitions.

[0101] Step 1002, the data processing apparatus determines the current query output of the at least two partitions according to the query algorithm.

[0102] In the embodiment, the data processing apparatus inputs the data records of the at least two partitions into the query algorithm respectively, and the output obtained is the current query output of the at least two partitions.

[0103] ​Step 1003, the data processing device determines whether there is a partition in which the current query output is the same as the historical query output in the at least two partitions.

[0104] In this embodiment, the data processing device stores the query output of each partition determined each time, which can be stored locally or in other external devices. The data processing device obtains the historical query output of each partition in the at least two partitions. For each partition, the data processing device determines whether the current query output of each partition is the same as the historical query output. For example, the input data set D includes at least two partitions D1 and D2, the current query outputs are O1 and O2 respectively, and the historical query outputs are O1' and O2' respectively. O1 and O1' are compared to determine whether they are the same, and O2 and O2' are compared to determine whether they are the same. If O1 and O1' are not the same, and O2 and O2' are not the same, then D1 and D2 must differ by two or more data records, and cannot constitute a differential attack, and the process proceeds to step 1004. If O1 and O1' are the same, and / or O2 and O2' are the same, then the current query output of the input data set and the historical query output can constitute a differential attack, and the process proceeds to step 1005.

[0105] It should be noted that since the partitions are divided when the data records are stored, subsequent updates to the input data set only update the data records in the partitions, and do not change the number of partitions in the input data set. Therefore, the current query output of a partition can not be the same as the historical query output.

[0106] Step 1004, if it is determined that the current query output of each partition in the at least two partitions is not the same as the historical query output, the data processing device performs the processing of step 603.

[0107] In this embodiment, when the data processing device determines that the current query output of each partition in the at least two partitions is not the same as the historical query output, the data processing device can determine that the input data set does not constitute a differential attack, and perform the processing of step 603, i.e., the data processing device can use the sensitivity to add noise to the first query output, obtain the first query output after adding noise, and send the first query output to the terminal used by the data queryer.

[0108] In addition, if there is no historical query output for a partition, the input data set does not constitute a differential attack, and the processing of step 603 can be directly performed.

[0109] In step 1005, the data processing apparatus determines, for a target partition of the at least two partitions, if the current query output of the target partition is the same as a historical query output of the target partition, deletes at least one data record in the target partition, so that the current query output of each of the at least two partitions is different from the historical query output, and executes the processes in steps 1006 to 1008.

[0110] In the embodiment, the target partition is a partition of the at least two partitions whose current query output is the same as a historical query output.

[0111] In the embodiment, the data processing apparatus determines that the current query output of the target partition is the same as a historical query output of the target partition, and can delete at least one data record in the target partition to satisfy that the current query output of the target partition is different from each historical query output.

[0112] It should be noted that, in order to make the query output provided for the data queryer relatively accurate, the at least one data record is one data record.

[0113] In step 1006, the data processing apparatus determines a query output after the target partition deletes the at least one data record according to a query algorithm.

[0114] In the embodiment, the data processing apparatus inputs the target partition that deletes the at least one data record into the query algorithm to obtain the query output. In addition, the data processing apparatus inputs data records in partitions other than the target partition of the at least two partitions into the query algorithm to obtain the query output.

[0115] In step 1007, the data processing apparatus determines a fourth query output of the input data set according to the query output after the target partition deletes the at least one data record.

[0116] In the embodiment, the data processing apparatus inputs the query output obtained in step 1006 into the query algorithm to obtain an output, which is the fourth query output of the input data set after the differential attack is prevented.

[0117] In step 1008, the data processing apparatus adds noise to the fourth query output according to the sensitivity, and outputs the fourth query output after the noise is added.

[0118] In the embodiment, the data processing apparatus can add noise to the fourth query output according to the sensitivity obtained in step 602 to obtain the fourth query output after the noise is added. The data processing apparatus sends the fourth query output after the noise is added to the terminal of the data queryer.

[0119] In this way, it can be accurately determined whether the differential attack is constituted, the differential attack can be prevented when the differential attack is constituted, and the personal data can be protected.

[0120] In addition, in the process of Figure 10 Step 602, the reason for using the sensitivity is that the data records in the input data set are relatively large, and the deletion of at least one data record in the target partition has a relatively small impact on the sensitivity, so it not only does not affect the noise added for the query output, but also saves the data processing time.

[0121] In addition, of course, in the embodiment of the present application, after step 1005 is executed, the sensitivity can also be recalculated according to the current input data set, and the recalculated sensitivity is also used in step 1008.

[0122] For the process of Figure 10 The following examples are provided for the process, as shown in Figure 11 The query algorithm is a summation algorithm, the input data set is {1, 4, 2, 3, 1, 2, 3, 1}, the first four data records of the input data set form partition one (i.e. D1 is {1, 4, 2, 3}), the last four data records of the input data set form partition two (i.e. D2 is {1, 2, 3, 1}), the current query output of partition one and partition two is 10 and 7, the historical query output of partition one is 9, 8, the historical query output of partition two is 100, 99, the current query output of partition one is not the same as the historical query output, the current query output of partition two is not the same as the historical query output, and the first query output corresponding to the input data set is 17.

[0123] As shown in Figure 12 The query algorithm is a summation algorithm, the input data set is {1, 4, 2, 3, 1, 2, 3, 1}, the first four data records of the input data set form partition one (i.e. D1 is {1, 4, 2, 3}), the last four data records of the input data set form partition two (i.e. D2 is {1, 2, 3, 1}), the current query output of partition one and partition two is 10 and 7, the historical query output of partition one is 10, 8, the historical query output of partition two is 100, 99, the current query output of partition one is the same as the historical query output, the data record 1 is deleted in partition one, the current query output changes from 10 to 9, the current query output of partition two is not the same as the historical query output, and the fourth query output corresponding to the input data set is 16 (i.e. 9+7=16).

[0124] When the present application is implemented on the Apache Spark big data processing system, as shown in Figure 13As shown, the sampling and partitioning module, the module for determining the query output of the partition and the sensitivity (referred to as the determining module) and the module for processing the query output (referred to as the output module) are included in Apache Spark. In the module for determining the sampling of the query output of the partition, the mapping operator and the reduction operator can be used. The sampling and partitioning module implements the partitioning and sampling, the determining module calculates the query output of the data records that are not sampled and belong to the first partition, the query output of the data records that are not sampled and belong to the second partition, the query output of the data records that are sampled and belong to the first partition, the query output of the data records that are sampled and belong to the second partition, and then uses the four query outputs to determine the query output of the sensitivity, the first partition and the second partition. The module for processing the query output uses the query output of the sensitivity, the first partition and the second partition to detect and prevent the differential attack and determine the first query output with the added noise mentioned above.

[0125] For example, corresponding to Figure 12 , the query algorithm is used for summation, the query algorithm is composed of the mapping operator M and the reduction operator R, the input data set is {1, 4, 2, 3, 1, 2, 3, 1}, the sampling and partitioning module obtains the input data set, divides the input data set D into the first partition (i.e. D1) and the second partition (i.e. D2), the data records that are not sampled form S', and the data records that are sampled form S. S'1 represents the data records that are not sampled and belong to D1, S'2 represents the data records that are not sampled and belong to D2, S1 represents the data records that are sampled and belong to D1, and S2 represents the data records that are sampled and belong to D2.

[0126] According to Figure 7 and Figure 11 , S'1 is {3}, S'2 is {1, 2, 3, 1}, S1 is {1, 4, 2}, and S2 is an empty set. The mapping operator in the determining module determines that M(S'1) = {3}, M(S'2) = {1, 2, 3, 1}, M(S1) = {1, 4, 2}, The mapping operator inputs M(S'1) = {3} and M(S1) = {1, 4, 2} to the first reduction operator in the determining module, and inputs to the second reduction operator in the determining module.

[0127] The first reduction operator in the determining module calculates the query output of M(S'1) as R(M(S'1)) = 3, the query output of M(S1) as R(M(S1)) = 7, and the first reduction operator determines R(M(S)), R(M(S-s1)),..., R(M(S-sn)). R(M(S-s1)) refers to the query output of the sampled data records after deleting the data record s1 from the sampled data records, and R(M(S-sn)) refers to the query output of the sampled data records after deleting the data record sn from the sampled data records, where n is the number of the sampled data records. The second reduction operator calculates the query output of M(S'2) as R(M(S'2)) = 7, and the query output of M(S2) as R(M(S2)) = 0. The first reduction operator and the second reduction operator input R(M(S'1)), R(M(S1)), R(M(S'2)), and M(S2) to the third reduction operator in the determining module, and the third reduction operator can obtain R(M(S')) = R(R(M(S'1)), R(M(S'2))) = 10 according to the commutativity and associativity.

[0128] R(M(S')) = R(R(M(S'1)), R(M(S'2))) = 10;

[0129] R(M(S)) = R(R(M(S1)), R(M(S2))) = 7;

[0130] R(M(D1)) = R(R(M(S'1)), R(M(S1))) = 10;

[0131] R(M(D2)) = R(R(M(S'2)), R(M(S2))) = 7.

[0132] And the third reduction operator can also determine the disturbance of each data record in the sampled data records to the input data set according to R(M(S')) and R(M(S)), i.e., R(M(D-s1)),..., R(M(D-sn)). R(M(D-s1)) refers to the disturbance of the query output to the input data set after deleting the data record s1 from the sampled data records, and R(M(D-sn)) refers to the disturbance of the query output to the input data set after deleting the data record sn from the sampled data records. Wherein, n is the number of the sampled data records, R(M(D-s1)) = R(M(D1)) + R(M(D2)) - R(M(S-s1)),..., R(M(D-sn)) = R(M(D1)) + R(M(D2)) - R(M(S-sn)).

[0133] Then the third reduction operator outputs R(M(D1)), R(M(D2)), R(M(D-s1)),..., R(M(D-sn)) to the module for processing the query output.

[0134] The module for processing query output compares R(M(D1)) with the historical query output of D1, and R(M(D2)) with the historical query output of D2. If R(M(D1)) is different from the historical query output of D1, and R(M(D2)) is different from the historical query output of D2, the module combines R(M(D1)) and R(M(D2)) to obtain the query output R(M(D)) of the input data set D. If R(M(D1)) is the same as the historical query output of D1, and R(M(D2)) is different from the historical query output of D2, the module deletes a data record in D1 so that the current query output of D1 is different from the historical query output, and then determines the query output R(M(D1)) of D1 after the deletion of the data record, combines R(M(D1)) after the deletion of the data record and R(M(D2)) to obtain the query output R(M(D)) of the input data set D. If R(M(D1)) is different from the historical query output of D1, and R(M(D2)) is the same as the historical query output of D2, the module deletes a data record in D2 so that the current query output of D2 is different from the historical query output, and then determines the query output R(M(D2)) of D2 after the deletion of the data record, combines R(M(D1)) and R(M(D2)) after the deletion of the data record to obtain the query output R(M(D)) of the input data set D. If R(M(D1)) is the same as the historical query output of D1, and R(M(D2)) is the same as the historical query output of D2, the module deletes a data record in D1 and D2 respectively so that the current query output of D1 after the deletion of the data record is different from the historical query output of D1, and the current query output of D2 after the deletion of the data record is different from the historical query output of D2. Then the module determines the query outputs R(M(D1)) and R(M(D2)) of D1 and D2 after the deletion of the data records, combines R(M(D1)) after the deletion of the data record and R(M(D2)) after the deletion of the data record to obtain the query output R(M(D)) of the input data set D.

[0135] The module for processing query output determines the maximum value in R(M(D-s1)),...,R(M(D-sn)), which is the sensitivity of the input data set D. Then the module adds Laplace noise to R(M(D)) to obtain the query output with the noise, and returns the query output with the noise to the terminal of the data queryer.

[0136] In addition, in order to more intuitively show the technical scheme of the present application, the data records in the input data set are relatively few in the actual processing, and the input data set includes a large number of data records.

[0137] In addition, in the embodiment of the present application, since the big data operators generally have commutativity and associativity, the embodiment of the present application can be applied to general big data query algorithms.

[0138] Figure 14 is a structural diagram of a data processing apparatus provided by the embodiment of the present application. The apparatus can be realized as part or all of a server by software, hardware or a combination of both. The server provided by the embodiment of the present application can implement the method provided by the embodiment of the present application. Figure 6 The flow, the apparatus comprises a determination module 1410 and an output module 1420, wherein:

[0139] The determination module 1410 is configured to:

[0140] According to the query algorithm, determine the first query output corresponding to the input data set;

[0141] According to the second query output of the unsampled data records in the input data set and the query algorithm, determine the perturbation of each data record in the target number of data records sampled in the input data set to the first query output, obtain the sensitivity corresponding to the input data set, and the unsampled data records and the target number of data records constitute the input data set; The determination function in the above step 601 and other implicit steps can be implemented;

[0142] The output module 1420 is configured to add noise to the first query output according to the sensitivity, and output the first query output after adding noise; The output function in the above step 602 and other implicit steps can be implemented.

[0143] In a possible implementation manner, as shown in Figure 15 The apparatus further comprises:

[0144] The receiving module 1430 is configured to, before determining the first query output corresponding to the input data set according to the query algorithm, receive a data query request, wherein the data query request comprises the query algorithm;

[0145] The sampling and partitioning module 1440 is configured to randomly sample the target number of data records in the input data set corresponding to the data query request, and obtain the target number of data records and the unsampled data records in the input data set.

[0146] In a possible implementation manner, the determination module 1410 is configured to:

[0147] According to the query algorithm, determine the second query output corresponding to the unsampled data records and the third query output corresponding to the target number of data records;

[0148] According to the second query output and the third query output, determine the first query output corresponding to the input data set.

[0149] In a possible implementation, the determining module 1410 is configured to:

[0150] According to the query algorithm, determine a third query output corresponding to the target number of data records;

[0151] According to the query algorithm, determine a query output after each data record in the target number of data records is deleted respectively;

[0152] According to the second query output of the unsampled data record, the third query output, and the query output after each data record in the target number of data records is deleted respectively, determine a perturbation of each data record in the target number of data records to the first query output;

[0153] Determine a maximum perturbation in the perturbations of the target number of data records to the first query output as a sensitivity corresponding to the input data set.

[0154] In a possible implementation, the sampling and partitioning module 1440 is further configured to:

[0155] According to the sensitivity, add noise to the first query output, and before outputting the first query output after adding noise, divide the input data set into at least two partitions according to the partitions to which the data records belong;

[0156] The determining module 1410 is further configured to:

[0157] According to the query algorithm, determine a current query output of the at least two partitions;

[0158] Determine that the current query output of each partition in the at least two partitions is different from a historical query output.

[0159] In a possible implementation, the determining module 1410 is further configured to:

[0160] For a target partition in the at least two partitions, if the current query output of the target partition is the same as a historical query output of the target partition, delete at least one data record in the target partition, so that the current query output of each partition in the at least two partitions is different from the historical query output.

[0161] According to the query algorithm, determine a query output after the target partition deletes the at least one data record;

[0162] determine a fourth query output of the input data set according to the query output after the at least one data record in the target partition is deleted;

[0163] The output module 1420 is further configured to add noise to the fourth query output according to the sensitivity, and output the fourth query output after the noise is added.

[0164] The division of the modules in the embodiments of the present application is illustrative, and is merely a logical function division. In actual implementation, another division manner can be used. In addition, the function modules in each embodiment of the present application can be integrated in one processor, or can be physically separated, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware, or in the form of a software function module.

[0165] In the above embodiments, all or part of the embodiments can be realized by software, hardware, firmware or any combination thereof. When the software is used for realization, all or part of the embodiments can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a server or terminal, all or part of the processes or functions described in the embodiments of the present application are generated. The computer instructions can be stored in a computer readable storage medium, or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a server or terminal, or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk and a magnetic tape, etc.), an optical medium (such as a digital video disk (Digital Video Disk, DVD), etc.), or a semiconductor medium (such as a solid state disk, etc.).

Claims

1. A method of data processing, characterized by, The method comprises: determining a first query output corresponding to an input data set according to a query algorithm; determining a third query output corresponding to a target number of data records sampled from the input data set according to the query algorithm; determining a query output after deleting each data record in the target number of data records respectively according to the query algorithm; determining a perturbation of each data record in the target number of data records on the first query output according to a second query output of a data record not sampled from the input data set, the third query output, and the query output after deleting each data record in the target number of data records respectively, wherein the second query output is obtained by processing the data record not sampled from the input data set based on the query algorithm; determining a maximum perturbation in the perturbations of the target number of data records on the first query output as a sensitivity corresponding to the input data set, wherein the data record not sampled from the input data set and the target number of data records constitute the input data set, the target number is greater than 1 and less than the number of data records in the input data set; adding noise to the first query output according to the sensitivity, and outputting a first query output after adding noise.

2. The method of claim 1, wherein, Before the determining a first query output corresponding to an input data set according to a query algorithm, the method further comprises: receiving a data query request, wherein the data query request comprises a query algorithm; randomly sampling a target number of data records from the input data set corresponding to the data query request, and obtaining the target number of data records and a data record not sampled from the input data set.

3. The method according to claim 1 or 2, characterized in that, The determining a first query output corresponding to an input data set according to a query algorithm comprises: determining a second query output corresponding to the data record not sampled from the input data set and a third query output corresponding to the target number of data records according to the query algorithm; determining the first query output corresponding to the input data set according to the second query output and the third query output.

4. The method according to claim 1 or 2, characterized in that, Before the adding noise to the first query output according to the sensitivity, and outputting a first query output after adding noise, the method further comprises: dividing the input data set into at least two partitions according to a partition to which a data record belongs; determining a current query output of the at least two partitions according to the query algorithm; determining that the current query output of each partition in the at least two partitions is different from a historical query output of the each partition.

5. The method of claim 4, wherein, The method further comprises: for a target partition in the at least two partitions, if the current query output of the target partition is the same as a historical query output of the target partition, deleting at least one data record in the target partition, so that the current query output of each partition in the at least two partitions is different from the historical query output of the each partition; determining a query output of the target partition after deleting the at least one data record according to the query algorithm; determining a fourth query output of the input data set according to the query output of the target partition after deleting the at least one data record; adding noise to the fourth query output according to the sensitivity, and outputting a fourth query output after adding noise.

6. An apparatus for data processing, characterized by The apparatus comprises: A determination module configured to: determine a first query output corresponding to an input data set according to a query algorithm; determine a third query output corresponding to a target number of data records sampled from the input data set according to the query algorithm; determine a query output after each data record in the target number of data records is deleted respectively according to the query algorithm; determine a perturbation of each data record in the target number of data records to the first query output according to a second query output of data records not sampled from the input data set, the third query output, and the query output after each data record in the target number of data records is deleted respectively, wherein the second query output is obtained by processing the data records not sampled from the input data set based on the query algorithm; determine a maximum perturbation in the perturbations of the target number of data records to the first query output as a sensitivity corresponding to the input data set, wherein the data records not sampled from the input data set and the target number of data records constitute the input data set, the target number is greater than 1 and less than a number of data records in the input data set; An output module configured to add noise to the first query output according to the sensitivity, and output the first query output after the noise is added.

7. The apparatus of claim 6, wherein, The apparatus further comprises: A receiving module configured to receive a data query request before determining a first query output corresponding to an input data set according to a query algorithm, wherein the data query request comprises the query algorithm; A sampling and partitioning module configured to randomly sample a target number of data records from the input data set corresponding to the data query request, and obtain the target number of data records and data records not sampled from the input data set.

8. The apparatus of claim 6 or 7, wherein, The determination module is configured to: determine a second query output corresponding to the data records not sampled from the input data set and a third query output corresponding to the target number of data records according to the query algorithm; determine the first query output corresponding to the input data set according to the second query output and the third query output.

9. The apparatus of any one of claims 6 or 7, wherein, The sampling and partitioning module is further configured to: divide the input data set into at least two partitions according to a partition to which a data record belongs before adding noise to the first query output according to the sensitivity, and output the first query output after the noise is added. The determination module is further configured to: determine a current query output of the at least two partitions according to the query algorithm; determine that the current query output of each partition in the at least two partitions is different from a historical query output.

10. The apparatus of claim 9, wherein, The determination module is further configured to: for a target partition in the at least two partitions, if the current query output of the target partition is the same as a historical query output of the target partition, delete at least one data record in the target partition, so that the current query output of each partition in the at least two partitions is different from the historical query output; determine a query output of the target partition after the at least one data record is deleted from the target partition according to the query algorithm; determine a fourth query output of the input data set according to the query output of the target partition after the at least one data record is deleted from the target partition. The output module is further configured to add noise to the fourth query output according to the sensitivity, and output the fourth query output after adding noise.

11. A computing device for data processing, characterized by The computing device comprises a processor and a memory, wherein: The memory stores computer instructions; The processor executes the computer instructions to implement the method of any one of claims 1-5.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, when the computer instructions in the computer readable storage medium are executed by a computing device, the computing device executes the method of any one of claims 1-5, or the computing device implements the function of the apparatus of any one of claims 6-10.

Images