Method and device for evaluating performance of privacy protection algorithm, equipment and storage medium

Abstract

The embodiment of the application discloses a privacy protection algorithm performance evaluation method and device, electronic equipment and storage medium, the method comprises the following steps: obtaining original data, and obtaining a plurality of target data obtained by performing privacy protection operations on the original data by using a plurality of privacy protection algorithms; determining an evaluation index used for performance evaluation of the plurality of privacy protection algorithms, and displaying a plurality of calculation units corresponding to the evaluation index; obtaining a target evaluation process created for the evaluation index based on the displayed plurality of calculation units; executing the target evaluation process according to the original data and the plurality of target data to obtain evaluation values of the plurality of privacy protection algorithms under the evaluation index; and determining performance evaluation results of the plurality of privacy protection algorithms based on the evaluation values of the plurality of privacy protection algorithms under the evaluation index. The application can meet the demand of performance evaluation of privacy protection algorithms in more scenarios, and improve the evaluation efficiency.

Description

Technical Field

[0001] The embodiments of the present invention relate to data processing technology, and more particularly to a method, apparatus, electronic device and storage medium for evaluating the performance of a privacy protection algorithm. Background Technology

[0002] The advent of the big data era has driven the digital transformation of enterprises, but it has also posed a significant threat to user privacy. Protecting user privacy is crucial for businesses; once user privacy information is leaked, it directly threatens users' property and personal safety, and causes incalculable economic losses and reputational damage to enterprises.

[0003] In response to the increasing frequency of privacy breaches, researchers have proposed numerous privacy protection algorithms in recent years. Faced with such a multitude of algorithms, selecting the appropriate one for data protection requires performance evaluation to measure the effectiveness of different algorithms in protecting the same content. Currently, most methods for evaluating the performance of privacy protection algorithms are fixed, meaning the evaluation metrics and procedures are standardized. However, performance requirements for privacy protection algorithms may differ across different application scenarios, rendering current evaluation methods inadequate for practical applications. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for evaluating the performance of privacy-preserving algorithms, which can meet the needs of performance evaluation of privacy-preserving algorithms in more scenarios and improve evaluation efficiency.

[0005] In a first aspect, the present invention provides a performance evaluation method for a privacy protection algorithm, comprising: Obtain raw data, and obtain multiple target data by performing privacy protection operations on the raw data using multiple privacy protection algorithms; The evaluation metrics used to evaluate the performance of the multiple privacy-preserving algorithms are determined, and the multiple computing units corresponding to the evaluation metrics are displayed. Obtain the target evaluation process created for the evaluation indicators based on the multiple computing units shown in the display; The target evaluation process is executed based on the original data and the multiple target data to obtain the evaluation values ​​of the multiple privacy protection algorithms under the evaluation indicators; Based on the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics, the performance evaluation results of the multiple privacy protection algorithms are determined.

[0006] Secondly, the present invention provides a performance evaluation device for a privacy protection algorithm, comprising: The data acquisition module is used to acquire raw data and to acquire multiple target data obtained by performing privacy protection operations on the raw data using multiple privacy protection algorithms. The display module is used to determine the evaluation metrics used to evaluate the performance of the multiple privacy protection algorithms and to display the multiple computing units corresponding to the evaluation metrics. The process acquisition module is used to acquire the target evaluation process created by the multiple computing units for the evaluation index based on the display. The execution module is used to execute the target evaluation process based on the original data and the multiple target data to obtain the evaluation values ​​of the multiple privacy protection algorithms under the evaluation index; The determination module is used to determine the performance evaluation results of the multiple privacy protection algorithms based on their evaluation values ​​under the evaluation metrics.

[0007] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a performance evaluation method for a privacy protection algorithm as described in any embodiment of the present invention.

[0008] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a performance evaluation method for a privacy protection algorithm as described in any embodiment of the present invention.

[0009] In this invention, raw data can be acquired, and multiple target data obtained by performing privacy protection operations on the raw data using multiple privacy protection algorithms. Evaluation metrics for the performance evaluation of the multiple privacy protection algorithms are determined, and multiple computational units corresponding to the evaluation metrics are displayed. A target evaluation process created based on the displayed computational units for the evaluation metrics is obtained. The target evaluation process is executed according to the raw data and the multiple target data to obtain the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics. Based on the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics, the performance evaluation results of the multiple privacy protection algorithms are determined. In other words, this invention allows users to participate in setting evaluation metrics and constructing evaluation processes. Since users can more intuitively understand the performance of the required privacy protection algorithms and the actual application scenarios where they want to apply the privacy protection algorithms, using user-set evaluation metrics and created evaluation processes to perform performance evaluations of privacy protection algorithms will better meet practical application needs. Furthermore, the implementation scheme that allows users to participate in the performance evaluation of privacy protection algorithms is highly scalable, saves relevant groups time and effort spent on algorithm evaluation and comparison, and improves evaluation efficiency. Attached Figure Description

[0010] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating the performance evaluation method for the privacy protection algorithm provided by the present invention; Figure 2 This is another flowchart illustrating the performance evaluation method for the privacy protection algorithm provided by the present invention; Figure 3a This is a data acquisition example diagram of the performance evaluation method for the privacy protection algorithm provided by the present invention; Figure 3b This is an example diagram illustrating the processing order of a computational unit in the performance evaluation method for the privacy protection algorithm provided by this invention. Figure 3c This is an example diagram of the target evaluation process of the performance evaluation method for the privacy protection algorithm provided by the present invention; Figure 3d This is a flowchart illustrating the conversion process from a process flow diagram to code for the performance evaluation method of the privacy protection algorithm provided by this invention. Figure 4 This is a schematic diagram of the performance evaluation device for the privacy protection algorithm provided by the present invention; Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0012] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0013] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0014] Figure 1 This is a flowchart illustrating a performance evaluation method for a privacy-preserving algorithm provided by the present invention. This method can be executed by a performance evaluation device for the privacy-preserving algorithm provided by the present invention, which can be implemented in software and / or hardware. In a specific embodiment, the device can be integrated into an electronic device, such as a computer or server. The following embodiments will illustrate this using the integration of the device into an electronic device as an example. Figure 1 The method may specifically include the following steps: Step 101: Obtain the original data and then obtain multiple target data obtained by performing privacy protection operations on the original data using multiple privacy protection algorithms.

[0015] The original data refers to data from databases requiring privacy protection, such as bank data involving customer privacy and identity information of individuals within relevant institutions. Privacy protection algorithms are systematic strategies and mechanisms for protecting sensitive data, primarily including federated learning, homomorphic encryption, differential privacy, and multi-party secure computation. In its implementation, this invention first acquires the original data, and then acquires multiple target data sets obtained by processing the original data using various privacy protection algorithms.

[0016] Step 102: Determine the evaluation metrics used to evaluate the performance of multiple privacy-preserving algorithms, and display the multiple computational units corresponding to the evaluation metrics.

[0017] The performance evaluation of privacy-preserving algorithms primarily assesses them from both usability and security perspectives. Evaluation metrics refer to specific definitions of a particular aspect of the evaluation target, derived through actual measurement. These metrics can be existing ones, such as Hausdorff distance, Kendall coefficient, and Jensen-Shannon divergence, or newly defined metrics tailored to user needs to achieve the desired evaluation effect. Computational units refer to the various types of computational units that break down the complete metric calculation process based on its characteristics. In practice, the user first selects an evaluation metric from multiple options and then displays the various computational units within that metric.

[0018] Step 103: Obtain the target evaluation process based on multiple computational units presented as evaluation metrics.

[0019] Specifically, based on the user's privacy protection requirements, after determining the evaluation indicators and the required computing units, the execution order of the computing units is determined according to the computing characteristics and configuration information of each computing unit. The configuration information can be the computing unit's own information parameters, such as the method name, the function to be executed, the input and output data types, and the number of parameters. After arranging the computing units, the target evaluation process created for the evaluation indicator is obtained.

[0020] Step 104: Perform the target evaluation process based on the original data and multiple target data to obtain the evaluation values ​​of multiple privacy protection algorithms under the evaluation indicators.

[0021] Specifically, the original data is subjected to a target evaluation process with multiple target data, each of which has its corresponding privacy protection algorithm. After the target evaluation process is executed, the evaluation values ​​of multiple privacy protection algorithms under evaluation metrics are obtained, such as the values ​​of Hausdorff distance, Kendall coefficient and Jensen-Shannon divergence.

[0022] Step 105: Based on the evaluation values ​​of multiple privacy protection algorithms under the evaluation metrics, determine the performance evaluation results of multiple privacy protection algorithms.

[0023] Specifically, the evaluation values ​​of multiple privacy-preserving algorithms under different evaluation metrics are combined, such as by averaging, calculating the variance, or weighting the values, to obtain a comprehensive evaluation value. The performance evaluation results of the multiple privacy-preserving algorithms are then determined by comparing the magnitude of the comprehensive evaluation value.

[0024] The present invention acquires raw data and multiple target data obtained by performing privacy protection operations on the raw data using multiple privacy protection algorithms; determines the evaluation metrics used to evaluate the performance of the multiple privacy protection algorithms and displays multiple computing units corresponding to the evaluation metrics; acquires a target evaluation process created based on the displayed multiple computing units for the evaluation metrics; executes the target evaluation process according to the raw data and the multiple target data to obtain the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics; and determines the performance evaluation results of the multiple privacy protection algorithms based on the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics. The present invention can meet the needs of performance evaluation of privacy protection algorithms in more scenarios and improves evaluation efficiency.

[0025] Figure 2 This is another flowchart illustrating the performance evaluation method for the privacy protection algorithm provided by this invention, as shown below. Figure 2 As shown, the method may include the following steps: Step 201: Obtain the original data and then obtain multiple target data obtained by performing privacy protection operations on the original data using multiple privacy protection algorithms.

[0026] Specifically, the task execution process in this step is as follows: Figure 3a As shown, the original data is first acquired, then multiple privacy protection algorithm tasks are decomposed from the set of privacy protection algorithms and submitted to the privacy protection algorithm execution engine for processing. After each privacy protection algorithm task is completed, the target data protected by the current task will be obtained. This target data, as the result of the privacy protection algorithm task, will also be acquired by this invention. That is, the original data file is used as input, the privacy protection algorithm is executed, and the target data file is output and stored.

[0027] Step 202: Obtain the newly created metrics from the preset page and determine the newly created metrics as the evaluation metrics used to evaluate the performance of multiple privacy protection algorithms.

[0028] The evaluation metrics can include Hausdorff distance, Kendall coefficient, and Jensen-Shannon divergence, among others.

[0029] Specifically, Hausdorff distance refers to the distance between proper subsets in a metric space. It is widely used to measure the similarity of points in two datasets and can spatially measure the degree of offset between two datasets before and after protection. The smaller the value of this metric, the worse the spatial distribution of the dataset is protected by the corresponding privacy protection mechanism, and thus the better the data availability. The calculation formula is as follows: As can be seen from the calculation formula, H1 and H2 are the same operation, the difference being that the comparison objects are swapped. Therefore, they can be considered the same operation, namely calculating the nearest distance from each position in dataset A to dataset B, only the input parameters are different. Next, set operations are performed, and the maximum value between the two resulting sets is the final result.

[0030] Kendall's coefficient is a statistical value used to measure the correlation between multiple random variables. In the evaluation of privacy protection algorithms, it can be used to assess the relative change in the number of accesses to two datasets at any two points before and after protection. The higher the value of this index, the stronger the correlation and the better the data availability is maintained. The calculation formula is as follows: Where "preserving order" means that the relative magnitude of the access frequency remains unchanged before and after privacy protection, and n represents the number of samples.

[0031] The Jensen-Shannon divergence is used to evaluate the change in access frequency before and after privacy protection. The larger the value of this index, the lower the correlation and the better the privacy protection is maintained. The calculation formula is shown below: Among them Indicates the Kullback-Leibler divergence. and These represent the access frequency distribution before and after privacy protection, respectively. The two KL operations in the formula can be considered the same operation, only the objects they process differ. Meanwhile... This is a set operation, representing the calculation of the average value of each element at a corresponding index in two sets, P and Q. In calculating the divergence and Kendall's coefficient, to minimize the impact of different implementation details in the algorithm, the visit distribution based on cluster centers is used as the statistical object. Therefore, clustering must be performed before the actual calculation, and this part of the operation can obviously be reused.

[0032] In practice, the preset page refers to the page that is pre-set to determine the evaluation indicators of privacy protection algorithms. Users can customize and create the required indicators (i.e., create new indicators) on the preset page. The new indicators can be used as the evaluation indicators for the performance evaluation of multiple privacy protection algorithms. The new indicators can include one or more indicators.

[0033] Step 203: Obtain the selected metrics from the candidate metrics from the preset page, and determine the selected metrics as the evaluation metrics used to evaluate the performance of multiple privacy protection algorithms.

[0034] In practice, the preset page includes several common performance evaluation metrics. Users select existing metrics from the preset page according to their needs and use them as the evaluation metrics for evaluating the performance of multiple privacy protection algorithms.

[0035] Step 204: Determine the common calculation units corresponding to the evaluation indicators.

[0036] For example, in the calculation of evaluation indicators, the general processing order of the calculation unit is as follows: Figure 3b As shown, it is mainly divided into three parts: data processing, single-step calculation, and set operation. Among them, when the calculation process of multiple evaluation indicators is broken down into multiple calculation units according to their characteristics, some common calculation units will appear. These identified calculation units are called common calculation units.

[0037] Step 205: Display the common computing unit.

[0038] Step 206: Determine the extended computing units created based on the common computing units shown.

[0039] Specifically, after determining the common calculation units to be displayed, the calculation units are expanded according to user needs to obtain a number of extended calculation units, so that the calculation units can work together to form a complete evaluation index calculation process.

[0040] Step 207: Determine the unit execution order based on the operations performed on the controls corresponding to the common computing unit and the extended computing unit, respectively.

[0041] The common computing unit and the extended computing unit each have corresponding controls. Specifically, users can use the controls corresponding to the common computing unit and the extended computing unit to sort these two types of computing units, and determine the execution order of each computing unit based on the sorting.

[0042] Step 208: Create a target evaluation process for the evaluation indicators based on the execution configuration information of the common computing units, the execution configuration information of the extended computing units, and the unit execution order.

[0043] Execution configuration information refers to the information parameters of the computing unit itself, such as the method name, the function to be executed, the input and output data types, and the number of parameters. Based on the execution configuration information of common computing units, the execution configuration information of extended computing units, and the unit execution order, a target evaluation process is created for the evaluation metric. For example, taking Hausdorff distance as the evaluation metric, the target evaluation process for Hausdorff distance can be as follows: Figure 3cAs shown, the process is divided into two parts performed simultaneously. The first part involves inputting the acquired raw data into the evaluation metric calculation process. This process reads the raw data file, retrieves the check-in information, inputs the original check-in points into a list, and then extracts the latitude and longitude coordinates of the raw data from the check-in information. These coordinates are then input into the raw dataset. Next, the shortest distance from each location in the raw dataset to the target dataset is calculated. The second part involves inputting the target data into the evaluation metric calculation process. This process reads the target data file, retrieves the check-in information, inputs the target check-in points into a list, and then extracts the latitude and longitude coordinates of the target data from the check-in information. These coordinates are then input into the target dataset. Next, the shortest distance from each location in the target dataset to the original dataset is calculated. Finally, the maximum value between the two distance sets is calculated and used as the output, yielding the Hausdorff distance value.

[0044] Step 209: Perform a target evaluation process based on the original data and multiple target data to obtain the evaluation values ​​of multiple privacy protection algorithms under the evaluation indicators.

[0045] Step 210: Determine the comprehensive evaluation value of each privacy protection algorithm based on the evaluation values ​​of each privacy protection algorithm under multiple evaluation metrics.

[0046] The evaluation metrics can include multiple metrics, and a comprehensive evaluation value for each privacy protection algorithm is determined based on these multiple evaluation values. For example, the privacy protection algorithms include Algorithm 1, Algorithm 2, and Algorithm 3, which respectively protect file X, resulting in file X1, file X2, and file X3. Simultaneously, there are evaluation metrics A, B, and C. The values ​​of Algorithm 1 under metrics A, B, and C can be calculated based on file X and X1, Algorithm 2 under metrics A, B, and C can be calculated based on file X and X2, and Algorithm 3 under metrics A, B, and C can be calculated based on file X and X3. The three evaluation values ​​are combined to obtain the comprehensive evaluation value for Algorithms 1, 2, and 3. The combination method can be averaging, weighting, etc.

[0047] Step 211: Determine the performance evaluation results of multiple privacy protection algorithms based on the comprehensive evaluation value of each privacy protection algorithm.

[0048] Specifically, the impact of the comprehensive evaluation value on the performance evaluation result is determined based on the calculation method of the comprehensive evaluation value, thereby determining the performance evaluation result of the privacy protection algorithm. For example, the smaller the Hausdorff distance value, the worse the privacy protection performance of the algorithm.

[0049] Step 212 shows the evaluation values ​​of each privacy protection algorithm under multiple evaluation metrics.

[0050] Step 213 presents the comprehensive evaluation value for each privacy protection algorithm.

[0051] Step 214 presents the performance evaluation results of multiple privacy protection algorithms.

[0052] It should be understood that, although Figure 2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0053] The present invention acquires raw data and multiple target data obtained by performing privacy protection operations on the raw data using multiple privacy protection algorithms; determines the evaluation metrics used to evaluate the performance of the multiple privacy protection algorithms and displays multiple computing units corresponding to the evaluation metrics; acquires a target evaluation process created based on the displayed multiple computing units for the evaluation metrics; executes the target evaluation process according to the raw data and the multiple target data to obtain the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics; and determines the performance evaluation results of the multiple privacy protection algorithms based on the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics. The present invention can meet the needs of performance evaluation of privacy protection algorithms in more scenarios and improves evaluation efficiency.

[0054] For example, Figure 3dThis is a flowchart illustrating the conversion process from flow chart to code for the performance evaluation method of the privacy protection algorithm provided by this invention. The execution process of the entire flowchart follows the order of the lines in the drawn flow chart, from the starting node to the ending node. When executing the code block represented by the current node, it must be ensured that all predecessor nodes that succeed this node have been executed. To vividly reflect the sequential relationship between the various computational units in the entire computational process, this process is analogous to a directed graph, characterized by only one node having an in-degree of 0 and one node having an out-degree of 0, thus uniquely determining the entry and exit points of the flowchart. Except for the entry node, the execution of other nodes is constrained by the starting node of the line connecting to that node. The vertices in the graph represent computational units, and the directed edges represent the sequential execution relationship of nodes and the direction of parameter passing. Only after the computation of the starting point is completed can the computation of its ending point be performed. Furthermore, the result returned by the execution of the preceding node will serve as a parameter value required for the computation of the subsequent node. The conversion from workflow to code is a dynamic process of executing code based on the method name and transmitted parameters corresponding to the computation unit. The object to be initialized is not determined in advance; therefore, the execution of the function corresponding to each step node must be performed through reflection. Before execution, each computation unit needs to know the specific parameter values ​​obtained by the method corresponding to that unit, and pass them according to the type of input parameters in each computation unit.

[0055] Figure 4 This is a schematic diagram of a performance evaluation device for the privacy-preserving algorithm provided by the present invention. This device is suitable for executing the performance evaluation method for the privacy-preserving algorithm provided by the present invention. Figure 4 As shown, the device may specifically include: The data acquisition module 401 is used to acquire raw data and acquire multiple target data obtained by performing privacy protection operations on the raw data using multiple privacy protection algorithms; The display module 402 is used to determine the evaluation metrics used to evaluate the performance of the multiple privacy protection algorithms and to display the multiple computing units corresponding to the evaluation metrics. The process acquisition module 403 is used to acquire the target evaluation process created by the multiple computing units for the evaluation index based on the display. Execution module 404 is used to execute the target evaluation process based on the original data and the multiple target data to obtain the evaluation values ​​of the multiple privacy protection algorithms under the evaluation index; The determination module 405 is used to determine the performance evaluation results of the multiple privacy protection algorithms based on the evaluation values ​​of the multiple privacy protection algorithms under the evaluation index.

[0056] In one embodiment, the display module 402 is specifically used for: Retrieve newly created metrics from a preset page, and determine these newly created metrics as the evaluation metrics used to assess the performance of the multiple privacy protection algorithms; or The selected metrics are obtained from the preset page and determined as the evaluation metrics used to evaluate the performance of the multiple privacy protection algorithms.

[0057] In one embodiment, the display module 402 is specifically used for: Determine the common calculation units corresponding to the evaluation indicators; The common computing unit is shown.

[0058] In one embodiment, the process acquisition module 403 is specifically used for: Determine the extended computing units created based on the common computing units shown; Obtain the target evaluation process created for the evaluation index based on the common computing unit and the extended computing unit.

[0059] In one embodiment, the common computing unit and the extended computing unit each have corresponding controls, and the process acquisition module 403 is specifically used for: The execution order of the units is determined based on the operations performed on the controls corresponding to the common computing unit and the extended computing unit, respectively. The target evaluation process is created for the evaluation index based on the execution configuration information of the common computing unit, the execution configuration information of the extended computing unit, and the execution order of the unit.

[0060] In one embodiment, the evaluation indicators include multiple metrics, and the determining module 405 is specifically used for: Based on the evaluation values ​​of each privacy protection algorithm under the multiple evaluation metrics, a comprehensive evaluation value for each privacy protection algorithm is determined. The performance evaluation results of the multiple privacy protection algorithms are determined based on the comprehensive evaluation value of each privacy protection algorithm.

[0061] In one embodiment, the device further includes: The evaluation value display module is used to display the evaluation value of each privacy protection algorithm under the multiple evaluation metrics; The comprehensive display module is used to display the comprehensive evaluation value of each privacy protection algorithm. The evaluation results display module is used to display the performance evaluation results of the multiple privacy protection algorithms.

[0062] The apparatus of this invention acquires raw data and multiple target data obtained by performing privacy protection operations on the raw data using multiple privacy protection algorithms; determines the evaluation metrics used to evaluate the performance of the multiple privacy protection algorithms and displays multiple computing units corresponding to the evaluation metrics; acquires a target evaluation process created based on the displayed multiple computing units for the evaluation metrics; executes the target evaluation process according to the raw data and the multiple target data to obtain the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics; and determines the performance evaluation results of the multiple privacy protection algorithms based on the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics. This invention can meet the needs of performance evaluation of privacy protection algorithms in more scenarios and improves evaluation efficiency.

[0063] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a performance evaluation method for the privacy protection algorithm provided in any of the above embodiments.

[0064] The present invention also provides a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements a performance evaluation method for the privacy protection algorithm provided in any of the above embodiments.

[0065] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing the electronic device of the present invention. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the invention.

[0066] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the computer system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0067] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0068] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined above in the system of this invention.

[0069] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0070] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0071] The modules and / or units described in this invention can be implemented in software or hardware. The described modules and / or units can also be housed in a processor; for example, a processor can be described as including a data acquisition module, a display module, a process acquisition module, an execution module, and a determination module. The names of these modules do not necessarily limit the module itself.

[0072] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include: Obtain the original data and multiple target data obtained by performing privacy protection operations on the original data using multiple privacy protection algorithms; determine the evaluation metrics used to evaluate the performance of the multiple privacy protection algorithms and display the multiple computing units corresponding to the evaluation metrics; obtain the target evaluation process created based on the displayed multiple computing units for the evaluation metrics; execute the target evaluation process according to the original data and multiple target data to obtain the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics; determine the performance evaluation results of the multiple privacy protection algorithms based on the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics.

[0073] According to the technical solution of this invention, raw data is acquired, and multiple target data are obtained by performing privacy protection operations on the raw data using multiple privacy protection algorithms. Evaluation metrics for performance evaluation of the multiple privacy protection algorithms are determined, and multiple computational units corresponding to the evaluation metrics are displayed. A target evaluation process is obtained based on the displayed computational units for the evaluation metrics. The target evaluation process is executed according to the raw data and the multiple target data to obtain the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics. Based on the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics, the performance evaluation results of the multiple privacy protection algorithms are determined. This invention can meet the needs of performance evaluation of privacy protection algorithms in more scenarios and improves evaluation efficiency.

[0074] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0075] The technical solution of this application complies with the relevant provisions of national laws and regulations regarding data acquisition, storage, use, and processing. The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A performance evaluation method for a privacy-preserving algorithm, characterized in that, include: Obtain raw data, and obtain multiple target data by performing privacy protection operations on the raw data using multiple privacy protection algorithms; The evaluation metrics used to evaluate the performance of the multiple privacy protection algorithms are determined, and multiple computational units corresponding to the evaluation metrics are displayed; wherein, the computational units are obtained by breaking down the complete metric calculation process according to the computational characteristics during the calculation of the evaluation metrics. Obtain the target evaluation process created for the evaluation indicators based on the multiple computing units shown in the display; The target evaluation process is executed based on the original data and the multiple target data to obtain the evaluation values ​​of the multiple privacy protection algorithms under the evaluation indicators; Based on the evaluation values ​​of the multiple privacy protection algorithms under the evaluation metrics, the performance evaluation results of the multiple privacy protection algorithms are determined.

2. The method according to claim 1, characterized in that, The evaluation metrics used to assess the performance of the multiple privacy-preserving algorithms include: Retrieve newly created metrics from a preset page, and determine these newly created metrics as the evaluation metrics used to assess the performance of the multiple privacy protection algorithms; or The selected metrics are obtained from the preset page and determined as the evaluation metrics used to evaluate the performance of the multiple privacy protection algorithms.

3. The method according to claim 1, characterized in that, The display shows multiple calculation units corresponding to the evaluation indicators, including: Identify the common calculation units corresponding to the evaluation indicators; wherein, the common calculation unit refers to the same calculation unit among the multiple calculation units obtained by breaking down the indicator calculation process of multiple evaluation indicators according to their calculation characteristics; The common computing unit is shown.

4. The method according to claim 3, characterized in that, The process of obtaining the target evaluation process created by the multiple computing units based on the display for the evaluation index includes: Determine the extended computing units created based on the common computing units shown; Obtain the target evaluation process created for the evaluation index based on the common computing unit and the extended computing unit.

5. The method according to claim 4, characterized in that, The common computing unit and the extended computing unit each have corresponding controls. The step of obtaining the target evaluation process created for the evaluation index based on the common computing unit and the extended computing unit includes: The execution order of the units is determined based on the operations performed on the controls corresponding to the common computing unit and the extended computing unit, respectively. The target evaluation process is created for the evaluation index based on the execution configuration information of the common computing unit, the execution configuration information of the extended computing unit, and the execution order of the unit.

6. The method according to claim 1, characterized in that, The evaluation metrics include multiple metrics. The process of determining the performance evaluation results of the multiple privacy-preserving algorithms based on their evaluation values ​​under these metrics includes: Based on the evaluation values ​​of each privacy protection algorithm under the multiple evaluation metrics, a comprehensive evaluation value for each privacy protection algorithm is determined. The performance evaluation results of the multiple privacy protection algorithms are determined based on the comprehensive evaluation value of each privacy protection algorithm.

7. The method according to claim 6, characterized in that, The method further includes: Show the evaluation values ​​of each privacy protection algorithm under the multiple evaluation metrics; and / or Show the comprehensive evaluation value of each privacy protection algorithm; and / or The performance evaluation results of the aforementioned privacy protection algorithms are presented.

8. A performance evaluation device for a privacy-preserving algorithm, characterized in that, include: The data acquisition module is used to acquire raw data and to acquire multiple target data obtained by performing privacy protection operations on the raw data using multiple privacy protection algorithms. The display module is used to determine the evaluation metrics used to evaluate the performance of the multiple privacy protection algorithms and to display the multiple computation units corresponding to the evaluation metrics; wherein, the computation units are obtained by breaking down the complete metric calculation process according to the computational characteristics during the calculation of the evaluation metrics. The process acquisition module is used to acquire the target evaluation process created by the multiple computing units for the evaluation index based on the display. The execution module is used to execute the target evaluation process based on the original data and the multiple target data to obtain the evaluation values ​​of the multiple privacy protection algorithms under the evaluation index; The determination module is used to determine the performance evaluation results of the multiple privacy protection algorithms based on their evaluation values ​​under the evaluation metrics.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a performance evaluation method for the privacy protection algorithm as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements a performance evaluation method for the privacy protection algorithm as described in any one of claims 1 to 7.

Images