Method, device, equipment and storage medium for model performance evaluation

Abstract

According to an embodiment of the present disclosure, a method, apparatus, device and storage medium for model performance evaluation are provided. The method includes, at a client node, determining a plurality of predicted classification results corresponding to a plurality of data samples by comparing a plurality of prediction scores output by a machine learning model for a plurality of data samples with a score threshold. The plurality of predicted classification results respectively indicate that the plurality of data samples are predicted to belong to a first category or a second category. The method also includes determining the values ​​of a plurality of metric parameters related to predetermined performance indicators of the machine learning model based on the differences between the plurality of predicted classification results and the plurality of true classification results corresponding to the plurality of data samples. The method also includes applying perturbations to the values ​​of the plurality of metric parameters to obtain perturbation values ​​of the plurality of metric parameters. The method also includes sending the perturbation values ​​of the plurality of metric parameters to a service node.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, and computer-readable storage media for model performance evaluation. Background Art

[0002] With the increasing importance of data privacy, current centralized machine learning systems are struggling to improve. This has led to the rise of federated learning. Federated learning can achieve performance comparable to traditional machine learning algorithms in an encrypted environment, even without leaving the local node. Federated learning leverages data from various nodes to achieve collaborative modeling and improve the effectiveness of artificial intelligence (AI) models, while ensuring data privacy. Federated learning allows individual nodes to remain on the same endpoint, achieving data protection. Federated learning is expected to better protect data privacy, including the privacy of the labels corresponding to data samples. Summary of the Invention

[0003] According to an example embodiment of the present disclosure, a solution for model performance evaluation is provided.

[0004] In a first aspect of the present disclosure, a method for model performance evaluation is provided. The method includes, at a client node, determining a plurality of predicted classification results corresponding to a plurality of data samples by comparing a plurality of prediction scores output by a machine learning model for a plurality of data samples with a score threshold. The plurality of predicted classification results respectively indicate that the plurality of data samples are predicted to belong to a first category or a second category. The method also includes determining the values ​​of a plurality of metric parameters related to predetermined performance indicators of the machine learning model based on the differences between the plurality of predicted classification results and the plurality of true classification results corresponding to the plurality of data samples. The method also includes applying perturbations to the values ​​of the plurality of metric parameters to obtain perturbation values ​​of the plurality of metric parameters. The method also includes sending the perturbation values ​​of the plurality of metric parameters to a service node.

[0005] In a second aspect of the present disclosure, a method for model performance evaluation is provided. The method includes receiving, at a service node, perturbation values ​​of multiple metric parameters related to a predetermined performance indicator of a machine learning model from client nodes of at least one group. The method also includes, for each of the at least one groups, aggregating the perturbation values ​​of the multiple metric parameters from the client nodes of the group by the metric parameter to obtain aggregated values ​​of the multiple metric parameters corresponding to the at least one group. The method also includes determining a value of the predetermined performance indicator based on at least one score threshold associated with the at least one group and the aggregated value of the at least one metric parameter corresponding to the at least one group.

[0006] In a third aspect of the present disclosure, a device for model performance evaluation is provided. The device includes a classification determination module, which is configured to determine a plurality of predicted classification results corresponding to a plurality of data samples by comparing a plurality of prediction scores output by a machine learning model for a plurality of data samples with a score threshold. The plurality of predicted classification results respectively indicate that a plurality of data samples are predicted to belong to a first category or a second category. The device also includes a metric parameter determination module, which is configured to determine the values ​​of a plurality of metric parameters related to predetermined performance indicators of the machine learning model based on the differences between the plurality of predicted classification results and the plurality of true classification results corresponding to the plurality of data samples. The device also includes a perturbation module, which is configured to apply perturbations to the values ​​of the plurality of metric parameters to obtain perturbation values ​​of the plurality of metric parameters. The device also includes a perturbation value sending module, which is configured to send the perturbation values ​​of the plurality of metric parameters to a service node.

[0007] In a fourth aspect of the present disclosure, a device for model performance evaluation is provided. The device includes a disturbance value receiving module configured to receive disturbance values ​​of multiple metric parameters related to predetermined performance indicators of a machine learning model from client nodes of at least one group. The device also includes an aggregation module configured to aggregate the disturbance values ​​of multiple metric parameters from the client nodes of the group according to the metric parameters for each group in at least one group, and obtain aggregate values ​​of multiple metric parameters corresponding to the at least one group. The device also includes an indicator determination module configured to determine the value of the predetermined performance indicator based on at least one score threshold associated with the at least one group and the aggregate value of the multiple metric parameters corresponding to the at least one group.

[0008] In a fifth aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0009] In a sixth aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the second aspect.

[0010] In a seventh aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and the computer program is executed by a processor to implement the method of the first aspect.

[0011] In an eighth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and the computer program is executed by a processor to implement the method of the second aspect.

[0012] It should be understood that the contents described in the summary of the present invention are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0014] Figure 1 A schematic diagram illustrating an example environment in which embodiments of the present disclosure can be applied;

[0015] Figure 2 A flowchart illustrating a signaling flow for model performance evaluation according to some embodiments of the present disclosure;

[0016] Figure 3 A flowchart illustrating a process of applying a perturbation according to some embodiments of the present disclosure;

[0017] Figure 4 A schematic diagram showing an ROC curve according to some embodiments of the present disclosure;

[0018] Figure 5 A flowchart illustrating a process of model performance evaluation according to some embodiments of the present disclosure;

[0019] Figure 6 A flowchart illustrating another process of model performance evaluation according to some embodiments of the present disclosure;

[0020] Figure 7 A block diagram illustrating another apparatus for model performance evaluation according to some embodiments of the present disclosure is shown;

[0021] Figure 8 A block diagram illustrating an apparatus for model performance evaluation according to some embodiments of the present disclosure; and

[0022] Figure 9 A block diagram is shown of a computing device / system capable of implementing one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0023] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0024] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below.

[0025] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0026] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0027] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0028] As an optional but non-limiting implementation, in response to receiving a user's active request, a prompt message may be sent to the user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. Furthermore, the pop-up window may also include a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0029] It is understandable that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0030] As used herein, the term "model" can learn the association between corresponding inputs and outputs from training data, so that after training is completed, corresponding outputs can be generated for given inputs. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. A neural network model is an example of a model based on deep learning. In this article, a "model" may also be referred to as a "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.

[0031] A "neural network" is a machine learning network based on deep learning. A neural network is capable of processing inputs and providing corresponding outputs. It typically includes an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications typically include many hidden layers, thereby increasing the depth of the network. The layers of a neural network are connected in sequence so that the output of the previous layer is provided as input to the next layer, where the input layer receives the input of the neural network and the output of the output layer serves as the final output of the neural network. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each of which processes the input from the previous layer.

[0032] Generally speaking, machine learning can be roughly divided into three stages, namely the training stage, the testing stage, and the application stage (also known as the inference stage). In the training stage, a given model can be trained using a large amount of training data, and the parameter values ​​are continuously updated iteratively until the model can obtain consistent inferences that meet the expected goals from the training data. Through training, the model can be considered to be able to learn the association between input and output (also known as input-to-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing stage, the test input is applied to the trained model to test whether the model can provide the correct output, thereby determining the performance of the model. In the application stage, the model can be used to process the actual input based on the parameter values ​​obtained through training to determine the corresponding output.

[0033] Figure 1A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. Environment 100 relates to a federated learning environment, which includes N client nodes 110-1...110-k,...110-N (where N is an integer greater than 1, k=1, 2,...N) and a service node 120. Client nodes 110-1...110-k,...110-N can respectively maintain respective local datasets 112-1...112-k,...112-N. For ease of discussion, client nodes 110-1...110-k,...110-N may be collectively or individually referred to as client nodes 110, and local datasets 112-1...112-k,...112-N may be collectively or individually referred to as local datasets 112.

[0034] In some embodiments, client node 110 and / or service node 120 can be implemented at terminal device or server.Terminal device can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phone, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, media computer, multimedia tablet, personal communication system (PCS) equipment, personal navigation device, personal digital assistant (PDA), audio / video player, digital camera / camcorder, positioning device, television receiver, radio broadcast receiver, e-book device, gaming device or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof.In some embodiments, terminal device can also support any type of interface for user (such as " wearable " circuit etc.).Server is various types of computing system / server that can provide computing power, including but not limited to mainframe, edge computing node, computing device in cloud environment, etc.

[0035] In federated learning, a client node is a node that provides part of the training data for a machine learning model. A client node can also be called a client, terminal node, terminal device, or user device. In federated learning, a service node is a node that aggregates the training results from client nodes.

[0036] exist Figure 1 In the example, assume that N client nodes 110 jointly participate in the training of the machine learning model 130 and aggregate the intermediate results of the training to the service node 120, so that the service node 120 can update the parameter set of the machine learning model 130. The entire set of local data of these client nodes 110 constitutes the complete training data set of the machine learning model 130. Therefore, according to the mechanism of federated learning, the service node 120 will generate a global machine learning model 130.

[0037] For the machine learning model 130 , the local dataset 112 at the client node 110 may include data samples and ground truth labels. Figure 1 Specifically shown is a local data set 112-k at a client node 110-k, which includes a data sample set and a true value label set. The data sample set includes a plurality of (M) data samples 102-1, ... 1102-i, ... 102-M (collectively or individually referred to as data samples 102), and the true value label set includes a corresponding plurality of (M) ground-truth labels 105-1, ... 1105-i, ... 105-M (collectively or individually referred to as true value labels 105). Wherein M is an integer greater than 1, i = 1, 2, ... M. Each data sample 102 can be annotated with a corresponding true value label 105. The data sample 102 can correspond to the input of the machine learning model 130, and the true value label 105 indicates the true output of the data sample 102. The true value label is an important part of supervised machine learning.

[0038] In an embodiment of the present disclosure, the machine learning model 130 can be constructed based on various machine learning or deep learning model architectures and can be configured to implement various prediction tasks, such as various classification tasks, recommendation tasks, etc. The machine learning model 130 can also be referred to as a prediction model, a recommendation model, a classification model, etc.

[0039] The data sample 102 may include input information related to a specific task of the machine learning model 130, and the true value label 105 is related to the expected output of the task. As an example, in a binary classification task, the machine learning model 130 may be configured to predict whether the input data sample belongs to the first category or the second category, and the true value label is used to mark whether the data sample actually belongs to the first category or the second category. Many practical applications can be classified as such binary classification tasks, such as whether the recommended item is converted (e.g., clicked, purchased, registered, or other demand behaviors) in a recommendation task, and so on.

[0040] It should be understood that Figure 1 The federated learning environment is shown as an example only. Depending on the federated learning algorithm and actual application needs, the environment can also be different. For example, although shown as a separate node, in some applications, the service node 120, in addition to serving as a central node, can also serve as a client node to provide some data for model training, model performance evaluation, etc. The embodiments of the present disclosure are not limited in this respect.

[0041] During the training phase of the machine learning model 130, mechanisms are in place to protect the local data of each client node 110 from being leaked. For example, during model training, the client node 110 does not need to disclose local data samples or label data. Instead, it sends gradient data calculated based on the local training data to the service node 120, allowing the service node 120 to update the parameter set of the machine learning model 130.

[0042] In some cases, it is also desirable to globally evaluate the performance of the trained machine learning model 130. The performance of the machine learning model 130 can be measured by one or more performance indicators. Different performance indicators can measure the difference between the predicted output given by the machine learning model 130 for the data sample set and the actual output indicated by the true value label set from different perspectives. Generally, if the difference between the predicted output given by the machine learning model 130 and the true output is small, it means that the performance of the machine learning model is better. It can be seen that it is usually necessary to determine the performance indicator of the machine learning model 130 based on the true value label set of the data sample.

[0043] As data regulation continues to strengthen, the demand for data privacy protection is becoming increasingly stringent, including the need to protect the true value labels of data samples to prevent them from being leaked. Therefore, determining the performance indicators of machine learning model 130 while protecting the local labeled data of client nodes from being leaked is a challenging task. Currently, there is no particularly effective solution to this problem.

[0044] According to an embodiment of the present disclosure, a model performance evaluation scheme is provided, which is capable of protecting the local label data of the client node. Specifically, at the client node, by comparing the multiple prediction scores output by the machine learning model for the multiple data samples with the score threshold received from the service node, the multiple prediction classification results corresponding to the multiple data samples are determined. The client node determines the values ​​of multiple metric parameters related to the predetermined performance indicators of the machine learning model based on the differences between the multiple prediction classification results and the multiple true value classification results corresponding to the multiple data samples. The client node applies perturbations to the values ​​of the multiple metric parameters to obtain the perturbation values ​​of the multiple metric parameters. The client node sends the perturbation values ​​of the multiple metric parameters to the service node. The service node determines the predetermined performance indicators based on the perturbation values ​​of the multiple metric parameters received from each client node.

[0045] According to the embodiments of the present disclosure, each client node does not need to expose its local true value label set or its local predicted classification results (i.e., predicted label information). At the same time, the service node can also calculate the value of the performance indicator based on the feedback information of the client node (for example, the perturbation value of multiple metric parameters). In this way, while determining the performance indicator of the machine learning model, the privacy of the client node's local label data is achieved.

[0046] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0047] Figure 2 A schematic diagram of a signaling flow 200 for model performance evaluation according to some embodiments of the present disclosure is shown. The signaling flow 200 involves a service node 120 and a plurality of client node groups 202-1, 202-2, ... 202-L in the environment 100. For ease of discussion, these client node groups are collectively or individually referred to herein as client node groups, where L is an integer greater than or equal to 1.

[0048] Client node group 202 may include multiple client nodes 110. For example, taking client node group 202-1 as an example, client node group 202-1 may include client nodes 110-1, 110-2, ..., 110-J, where J is an integer greater than or equal to 1 and less than or equal to N. It should be understood that signaling flow 200 may involve any number of service nodes 120 and any number of client node groups 202.

[0049] It should be understood that each client node group 202 may include any number of client nodes 110. The number of client nodes 110 included in each client node group 202 may be the same or different. In some embodiments, N client nodes 110 may be evenly or approximately evenly divided into L client node groups 202, where each client node group 202 includes N / L (an integer) number of client nodes 110.

[0050] In an embodiment of the present disclosure, it is assumed that the performance of the machine learning model 130 is to be evaluated. In some embodiments, the machine learning model 130 to be evaluated may be a global machine learning model determined based on a training process of federated learning, for example, the client node 110 and the service node 120 participate in the training process of the machine learning model 130. In some embodiments, the machine learning model 130 may also be a model obtained in any other manner, and the client node 110 and the service node 120 may not participate in the training process of the machine learning model 130. The scope of the present disclosure is not limited in this respect.

[0051] In some embodiments, the service node 120 sends (not shown) the machine learning model 130 to each client node 110 in the client node group 202. After receiving the machine learning model 130, each client node 110 may perform a subsequent evaluation process based on the machine learning model 130. In some embodiments, the machine learning model 130 to be evaluated may also be provided to the client node 110 in any other appropriate manner.

[0052] In some embodiments, the service node 120 may send multiple score thresholds to the client nodes 110 in at least one client node group 202. For example, the service node 120 may randomly generate L score thresholds and send the L score thresholds to each client node 110 in the L client node groups 202. Each score threshold is a value between 0 and 1.

[0053] In some embodiments, the value of L (i.e., the number of score thresholds or the number of client node groups 202) can be predetermined by the service node 120. For example, the value of L can be determined based on the number of client nodes 110. In some embodiments, the value of L can also be determined based on the type of predetermined performance indicator to be determined by the service node 120. For example, if the predetermined performance indicator to be determined by the service node 120 is the accuracy of the prediction result (ACC), the service node 120 can determine the value of L to be 1. For another example, if the predetermined performance indicator to be determined by the service node 120 is the area under the curve (AUC) of the receiver operating characteristic curve (ROC), the service node 120 can determine the value of L to be an integer greater than 1. It should be understood that for the case where the predetermined performance indicator is ACC or the AUC of ROC, the value of L can also be determined as other appropriate integer values. The embodiments of the present disclosure are not limited in this respect.

[0054] like Figure 2 As shown, the service node 120 may send (205-1) a first score threshold to the client node 110 in the client node group 202-1 associated therewith. Similarly, the service node 120 may send (205-2 / ... / 205-L) a second / ... / Lth score threshold to the client node 110 in the client node group 202-2 / ... / 202-L associated therewith.

[0055] The client nodes 110 in each client node group 202 receive (210-1 / 210-2 / ... / 210-L) their respective score thresholds.

[0056] Taking the client node 110 in the client node group 202-1 as an example, the client node 110 may apply each data sample 102 to the machine learning model 130 as an input to the model, and obtain a prediction score output by the machine learning model 130. The client node 110 determines (215-1) a plurality of predicted classification results corresponding to the plurality of data samples 102 by comparing the plurality of predicted scores output by the machine learning model 130 for the plurality of data samples 102 with a score threshold. The plurality of predicted classification results respectively indicate that the plurality of data samples 102 are predicted to belong to the first category or the second category.

[0057] In the embodiments of the present disclosure, particular attention is paid to the performance indicators of machine learning models in implementing binary classification tasks. Each prediction score can indicate the predicted probability that the corresponding data sample 102 belongs to the first category or the second category. The two categories can be configured according to the actual task requirements.

[0058] The value range of the prediction score output by the machine learning model 130 can be set arbitrarily. For example, the prediction score can be a value in a certain continuous value interval (for example, a value between 0 and 1), or it can be a value in a plurality of discrete values ​​(for example, it can be one of the discrete values ​​such as 0, 1, 2, 3, 4, 5, etc.). In some examples, a higher prediction score can indicate that the predicted probability of the data sample 102 belonging to the first category is greater, and the predicted probability of belonging to the second category is smaller. Of course, the opposite setting is also possible, for example, a higher prediction score can indicate that the predicted probability of the data sample 102 belonging to the second category is greater, and the predicted probability of belonging to the first category is smaller.

[0059] In some embodiments, if the prediction score of the machine learning model 130 for the data sample 102 exceeds the score threshold, the client node 110 may determine the predicted classification result corresponding to the data sample 102 as indicating that the data sample belongs to the first category. Conversely, if the prediction score of the machine learning model 130 for the data sample 102 does not exceed the score threshold, the client node 110 may determine the predicted classification result corresponding to the data sample 102 as indicating that the data sample belongs to the second category.

[0060] In some embodiments, each data sample 102 has a true value label 105. The true value label 105 is used to mark whether the corresponding data sample 102 belongs to the first category or the second category. In the following, for the convenience of discussion, the data samples belonging to the first category are sometimes referred to as positive samples, positive examples or positive samples, and the data samples belonging to the second category are sometimes referred to as negative samples, negative examples or negative samples. In some embodiments, each true value label 105 can have one of two values, which are used to indicate the first category or the second category respectively. In some embodiments below, for the convenience of discussion, the value of the true value label 105 corresponding to the first category can be set to "1", which indicates that the corresponding data sample 102 belongs to the first category and is a positive sample. In addition, the value of the true value label 105 corresponding to the second category can be set to "0", which indicates that the corresponding data sample 102 belongs to the second category and is a negative sample.

[0061] It should be understood that the first category and the second category can be any category in a binary classification problem. Taking the binary classification problem of determining whether the content of an image is a cat as an example, the first category can represent the category of the content of the image being a cat, and the second category can represent the category of the content of the image being a non-cat. Taking the evaluation of the quality of an item as an example, the first category can represent that the quality of the item meets the standard, and the second category can represent that the quality of the item does not meet the standard. It should be understood that the binary classification problems listed above are merely exemplary, and the model performance evaluation method described herein is applicable to all types of binary classification problems. The embodiments of the present disclosure are not limited in this respect. In some example embodiments below, for ease of discussion, image classification is mainly used as an example for explanation, but it should be understood that this does not imply that those embodiments can only be applied to this binary classification problem.

[0062] The client nodes 110 in the client node group 202-1 determine (220-1) values ​​of a plurality of metric parameters associated with a predetermined performance indicator of the machine learning model 130 based on differences between the plurality of predicted classification results and a plurality of true classification results corresponding to the plurality of data samples 102. The plurality of true classification results may be annotated with a plurality of true labels of the plurality of data samples 102, respectively, to indicate whether the plurality of data samples 102 belong to the first category or the second category.

[0063] In some embodiments, the plurality of metric parameters may include a first number of first-category data samples in the plurality of data samples 102. Both the predicted classification result and the true classification result corresponding to the first-category data sample indicate the first category. For example, for a certain data sample 102, if the true classification result (or true value label 105) of the data sample 102 indicates that the data sample 102 belongs to the first category (e.g., an image of a cat), and the predicted classification result predicted by the machine learning model 130 also indicates that the data sample 102 belongs to the first category, then the data sample 102 belongs to the first category of data samples, also known as a true positive sample (TP).

[0064] In some embodiments, the plurality of metric parameters may include a second number of data samples of the second category in the plurality of data samples 102. Both the predicted classification result and the true classification result corresponding to the second category data samples indicate the second category. For example, for a certain data sample 102, if the true classification result (or true value label 105) of the data sample 102 indicates that the data sample 102 belongs to the second category (e.g., not an image of a cat), and the predicted classification result predicted by the machine learning model 130 also indicates that the data sample 102 belongs to the second category, then the data sample 102 belongs to the second category data sample, also known as a true negative sample (TN).

[0065] In some embodiments, the plurality of metric parameters may include a third number of data samples of the third category in the plurality of data samples 102. The predicted classification result corresponding to the data sample of the third category indicates the first category and the corresponding true classification result indicates the second category. For example, for a certain data sample 102, if the true classification result (or true value label 105) of the data sample 102 indicates that the data sample 102 belongs to the second category (e.g., not an image of a cat), and the predicted classification result predicted by the machine learning model 130 indicates that the data sample 102 belongs to the first category (e.g., an image of a cat), then the data sample 102 belongs to the third category of data samples, also known as a false positive sample (FP).

[0066] In some embodiments, the plurality of metric parameters may include a fourth number of data samples of the fourth category in the plurality of data samples. The predicted classification result corresponding to the fourth category data sample indicates the second category and the corresponding true value classification result indicates the first category. For example, for a certain data sample 102, if the true value classification result (or true value label 105) indicates that the data sample 102 belongs to the first category (for example, an image of a cat), and the predicted classification result predicted by the machine learning model 130 indicates that the data sample 102 belongs to the second category (for example, an image that does not belong to a cat), then the data sample 102 belongs to the fourth category data sample, also known as a false negative sample (False Negative, FN).

[0067] The four results described above are summarized in Table 1 below.

[0068] Table 1

[0069]

[0070] The above examples of multiple metric parameters are given using the first number (number of TPs), the second number (number of TNs), the third number (number of FPs), and the fourth number (number of FNs). It should be understood that the client node 110 may determine (220-1) the value of at least one of the above metric parameters based on the differences between the multiple predicted classification results and the multiple true classification results corresponding to the multiple data samples 102. Alternatively, the client node 110 may determine (220-1) the values ​​of the numbers of TPs, TNs, FPs, and FNs described above based on the above differences. Additionally, the client node 110 may also determine the values ​​of other additional metric parameters.

[0071] In the example where the client node 110 determines the values ​​of TP, TN, FP, and FN, the client node 110 may represent these four values ​​as a four-tuple, i.e., (FP, FN, TP, TN). Additionally, in some embodiments, the four values ​​may be stored together with the score threshold of the client node 110, e.g., represented as (k_i, FP, FN, TP, TN), where k_i represents the score threshold of the i-th client node group.

[0072] There are no duplicate client nodes 110 between the client node groups 202 divided as described above. In other words, the client nodes 110 in each client node group 202 within the plurality of client node groups 202 are distinct from the client nodes in the other client node groups 202. In this manner, each client node 110 receives only one score threshold. Each client node 110 determines the value of each metric parameter based on only one score threshold. In this manner, leakage of client node 110 information (such as predicted classification results or predicted classification labels) can be avoided.

[0073] The client node 110 in the client node group 202-1 applies perturbations to the values ​​of the plurality of metric parameters to obtain (225-1) perturbed values ​​of the plurality of metric parameters. For example, for at least one of TP, TN, FP, and FN, the client node 110 may add random perturbations to the one or more values ​​of TP, TN, FP, and FN using, for example, a Gaussian mechanism or a Laplace mechanism.

[0074] Figure 3 FIG. 3 is a flow chart illustrating a process 300 of applying perturbations according to some embodiments of the present disclosure. The process 300 may be implemented at the client node 110 .

[0075] Specifically, at block 310, client node 110 is configured to determine a sensitivity value associated with the disturbance. For example, sensitivity Δ may be set to 1. That is, the maximum impact on the statistic is 1 for each change in the label of data sample 102. Alternatively, the sensitivity value may be set to other appropriate values.

[0076] At block 320 , the client node 110 is configured to determine a random perturbation distribution based on the sensitivity value Δ and the label differential privacy mechanism.

[0077] The random response mechanism is a type of differential privacy (DP) mechanism. To better understand the embodiments of the present disclosure, the following will first briefly introduce differential privacy and the random response mechanism.

[0078] Assume that ∈, δ is a real number greater than or equal to 0, that is And M is a random mechanism (random algorithm). A random mechanism means that for a specific input, the output of the mechanism is not a fixed value, but follows a certain distribution. For a random mechanism M, it can be considered to have (∈, δ)-differential privacy if the following conditions are met: for any two adjacent training data sets D, D′: , and for any subset S of M's possible outputs, there exists:

[0079] Pr[M(D)∈S]≤e ∈ ·Pr[M(D′)∈S]+δ (1)

[0080] In addition, if δ = 0, the random mechanism M can also be considered to have ∈-differential privacy (∈-DP). In the differential privacy mechanism, for a random mechanism M with (∈, δ)-differential privacy or ∈-differential privacy, it is expected that the distributions of the two outputs obtained after it acts on two adjacent data sets are difficult to distinguish. In this way, the observer can hardly detect slight changes in the input data set of the algorithm by observing the output results, thereby achieving the purpose of privacy protection. If the random mechanism M acts on any adjacent data set and the probability of obtaining a specific output S is similar, then it will be considered that the algorithm is difficult to achieve the effect of differential privacy.

[0081] In the embodiments of this paper, we focus on the differential privacy of the labels of data samples, and the labels indicate the binary classification results. Therefore, following the setting of differential privacy, we can define label differential privacy. Specifically, assuming ∈, δ is a real number greater than or equal to 0, that is, And M is a random mechanism (randomized algorithm). A random mechanism M is considered to have (∈, δ)-label differential privacy if the following conditions are met: for any two adjacent training datasets D, D′:, they differ only in the label of a single data sample, and for any subset S of the possible outputs of M, there exists:

[0082] Pr[M(D)∈S]≤e ∈ ·Pr[M(D′)∈S]+δ (2)

[0083] Furthermore, if δ = 0, the random mechanism M can also be considered to have ∈-differential privacy (∈-DP). That is, it is expected that after changing the label of a data sample, the distribution of the output results from the random mechanism M remains small, making it difficult for observers to detect the change in the label.

[0084] The random response mechanism is a random mechanism applied to achieve differential privacy. The random response mechanism is positioned as follows: Assume ∈ is a parameter and y∈[0,1] is the known value of the true value label in the random response mechanism. If for the value y of the true value label, the random response mechanism derives a random value from the following probability distribution

[0085]

[0086] That is, after applying the random response mechanism, the random value There is a certain probability that it is equal to y, and there is also a certain probability that it is not equal to y. The above random response mechanism is considered to have label differential privacy with δ = 0 ((∈, 0)-label differential privacy) because:

[0087]

[0088] That is, the random response mechanism will satisfy ∈-differential privacy.

[0089] The above discusses differential privacy and random response mechanisms. Through the random response mechanism, the client node 110 can add random perturbations to the values ​​of multiple metric parameters to prevent the service node 120 from obtaining the privacy of the client node 110 (eg, predicted label information, etc.).

[0090] In some embodiments, the label differential privacy mechanism may be a Gaussian mechanism. The standard deviation σ of the random perturbation distribution (∈, δ)-DP of the Gaussian mechanism (i.e., the standard deviation of the added noise) can be calculated as the following equation (5).

[0091]

[0092] Here, Δ represents the sensitivity value, and δ and ∈ are arbitrary values ​​between 0 and 1 (excluding 0 and 1).

[0093] Alternatively, in some embodiments, the label differential privacy mechanism can be a Laplace mechanism. The random perturbation distribution of the Laplace mechanism satisfies (∈, 0)-DP, the scale of the Laplace distribution is b = Δ / ∈, and the standard deviation σ of the added random noise can be calculated as the following equation (6).

[0094]

[0095] It should be understood that the Gaussian mechanism and Laplace mechanism listed above are merely illustrative and not restrictive. The embodiments of the present disclosure may use other suitable label differential privacy mechanisms to determine the random perturbation distribution. The embodiments of the present disclosure are not limited in this respect.

[0096] At block 330, client node 110 is configured to apply a perturbation to at least one number based on a random perturbation distribution. For example, in the example where client node 110 determines the four values ​​of TP, TN, FP, and FN, client node 110 may apply a random perturbation to each of these four values ​​to obtain a perturbed value (FP', FN', TP', TN'), or alternatively, (k_i, FP', FN', TP', TN'). Alternatively, in some embodiments, client node 110 may apply a random perturbation to one or more of the four values.

[0097] By perturbing the value of the metric parameter, the information of the client node 110, especially the positive label information, can be prevented from being leaked. Specifically, this method of applying perturbations can prevent the service node 120 from inferring the predicted label of the client node 110. For example, if the value of the metric parameter is not perturbed, when TP / (TP+FP) is too large, the service node may determine that all samples greater than the threshold k_i are positive samples. For another example, if the value of the metric parameter is not perturbed, when FN / (FN+TN) is too large, the service node may determine that all samples less than the threshold k_i are negative samples. By perturbing the value of the metric parameter, the above-mentioned situation and other potential information leakage situations can be avoided.

[0098] Continue to refer Figure 2 The client node 110 in the client node group 202-1 sends (230-1) the perturbation values ​​of the plurality of metric parameters to the service node 120 for use in determining a predetermined performance indicator at the service node 120. In this document, "perturbation" is sometimes also referred to as noise, interference, etc.

[0099] In some embodiments, each client node 110 in the client node group 202-1 may send the four-tuple (FP', FN', TP', TN') or (k_i, FP', FN', TP', TN') to the service node 120. Alternatively, the client nodes 110 in the client node group 202-1 may send one or more of the four perturbation values ​​to the service node 120. The process of determining the predetermined performance indicator by the service node 120 will be described below.

[0100] The above description is based on the client node group 202-1 as an example. Similarly, the client nodes 110 in the client node group 202-2 / … / 202-L determine (215-2 / … / 215-L) a plurality of predicted classification results corresponding to the plurality of data samples 102 by comparing the plurality of predicted scores output by the machine learning model 130 for the plurality of data samples 102 with a score threshold. The client nodes 110 in the client node group 202-2 / … / 202-L determine (220-2 / … / 220-L) a plurality of metric parameters related to predetermined performance indicators of the machine learning model 130 based on the differences between the plurality of predicted classification results and the plurality of true classification results corresponding to the plurality of data samples 102. The client nodes 110 in the client node group 202-2 / … / 202-L apply perturbations to the values ​​of the plurality of metric parameters to obtain (225-2 / … / 225-L) a plurality of perturbation values ​​of the metric parameters. The client nodes 110 in the client node group 202-2 / ... / 202-L send (230-2 / ... / 230-L) the perturbed values ​​of the plurality of metric parameters to the service node 120 for use in determining the predetermined performance indicator at the service node 120. The above process is similar to the corresponding process of the client node group 202-1 and will not be further described here.

[0101] At the service node 120, perturbation values ​​of a plurality of metric parameters related to a predetermined performance indicator of the machine learning model 130 are received (235-1, or also including 235-2 / ... / 235-L, collectively referred to as 235) from at least one client node group 202-1 (or 202-2 / ... / 202-L) respectively. For example, the service node 120 may receive 235 a quadruple (FP', FN', TP', TN') or (k_i, FP', FN', TP', TN') respectively from at least one client node group 202. Alternatively, the service node 120 may receive (235) one or more of the four perturbation values, which may depend on the performance indicator to be calculated.

[0102] TP' represents a first perturbation number of data samples of a first class in the plurality of data samples 102 at a given client node 110, wherein the data samples of the first class are labeled as the first category by the true value label and are predicted as the first category by the machine learning model 130. TN' represents a second perturbation number of data samples of a second class in the plurality of data samples 102, wherein the data samples of the second class are labeled as the second category by the true value label and are also predicted as the second category by the model. FP' represents a third perturbation number of data samples of a third class in the plurality of data samples 102, wherein the data samples of the third class are labeled as the second category and are predicted as the first category. FN' represents a fourth perturbation number of data samples of a fourth class in the plurality of data samples 102, wherein the data samples of the fourth class are labeled as the first category and are predicted as the second category.

[0103] For each client node group in at least one client node group 202, the service node 120 aggregates the perturbation values ​​of multiple metric parameters of the client nodes 110 in the client node group according to the metric parameters to obtain aggregated values ​​of the multiple metric parameters corresponding to the at least one client node group 202. For example, the service node 120 aggregates (240-1) the perturbation values ​​(FP', FN', TP', TN') of the multiple metric parameters of the client nodes 110 in client node group 202-1 according to the metric parameters to obtain aggregated values ​​of the multiple metric parameters corresponding to client node group 202-1. Similarly, the service node 120 aggregates (240-2 / ... / 240-L) the perturbation values ​​(FP', FN', TP', TN') of the multiple metric parameters of the client nodes 110 in client node groups 202-2 / ... / 202-L according to the metric parameters to obtain aggregated values ​​of the multiple metric parameters corresponding to client node groups 202-2 / ... / 202-L.

[0104] In some embodiments, for each client node group 202, the service node 120 may calculate the aggregated values ​​TPR (True Positive Sample Rate) and FPR (False Positive Sample Rate) of multiple metric parameters corresponding to the client node group 202 based on the following equations (7) and (8). TPR represents the proportion of samples that are actually positive (positive samples) that are correctly judged as positive. FPR represents the proportion of samples that are actually negative (negative samples) that are incorrectly judged as positive.

[0105] TPR=TP′ / (TP′+FN′) (7)

[0106] FPR=FP′ / (FP′+TN′) (8)

[0107] It should be understood that the aggregate values ​​TPR and FPR described above are merely exemplary and not restrictive. In some embodiments, the service node 120 may use other methods to derive aggregate values ​​of multiple metric parameters corresponding to the client node group 202.

[0108] The service node 120 determines ( 245 ) a value of a predetermined performance indicator based on a plurality of score thresholds respectively associated with the at least one client node group 202 and an aggregate value of a plurality of metric parameters respectively corresponding to the at least one client node group 202 .

[0109] In some embodiments, the predetermined performance metric includes at least the area under the curve (AUC) of a receiver operating characteristic (ROC) curve. The service node 120 may determine the ROC of the machine learning model 130 based on at least one score threshold and an aggregate value of multiple metric parameters.

[0110] In some embodiments, the value of L (i.e., the number of score thresholds or the number of client node groups 202) may be greater than 1. In other words, the at least one group may include multiple groups and the at least one score threshold may include multiple score thresholds. The service node 120 may calculate the coordinate points of multiple (FPR, TPR) pairs based on each threshold score, and connect these points into a line to fit the ROC curve of the machine learning model 130.

[0111] The service node 120 may then determine the area under the ROC curve (AUC). By definition, AUC refers to the area under the ROC curve. In some embodiments, AUC may be calculated by using an approximate algorithm to calculate the area under the ROC curve, based on the definition of AUC.

[0112] Figure 4 Schematic diagram of ROC curve 410 according to some embodiments of the present disclosure is shown. ROC curve 410 is drawn by calculating the coordinate points of multiple (FPR, TPR) pairs according to multiple threshold scores (in this example, the value of L is greater than 1). Figure 4 In the example, the multiple coordinate points are (0,0), (0,0.2), (0.2,0.2), (0.2,0.4), (0.4,0.4), (0.4,0.6), (0.6,0.6), (0.6,0.8), (0.8,0.8), (0.8,1), and (1,1). Figure 4 4 shows a curve 420 that divides the ROC plane into two parts. The ROC curve 410 can be used to determine that the AUC of the ROC curve is 0.7. Alternatively, the area between the ROC curve 410 and the curve 420 can be determined to be 0.2, thereby determining that the AUC of the ROC curve 410 is 0.7.

[0113] It should be understood that Figure 4 The depicted ROC curve 410 is merely illustrative. Figure 4 The coordinate values ​​of the coordinate points in are for illustrative purposes only and are not intended to be limiting. The service node 120 may determine coordinate values ​​of other values ​​and may draw ROC curves of other shapes.

[0114] By dividing the client nodes 110 into two or more groups and calculating the coordinate points of multiple (FPR, TPR) pairs according to multiple threshold scores to obtain the AUC of the ROC, the result of the model performance evaluation can be made more accurate.

[0115] Additionally or alternatively, when calculating AUC, AUC can also be determined from a probabilistic perspective. AUC can be considered as the probability that, when a positive sample and a negative sample are randomly selected, the machine learning model gives the positive sample a higher prediction score than the negative sample. That is, in the data sample set, the positive and negative samples are combined into positive-negative sample pairs, in which the prediction score of the positive sample is greater than the prediction score of the negative sample. If the model can output a higher prediction score for more positive samples than the negative samples, it can be considered that the AUC is higher and the model performance is better. The value range of AUC is between 0.5 and 1. The closer the AUC is to 1, the better the performance of the model. In this example, the service node 120 can determine the AUC from a probabilistic perspective based on the corresponding score threshold of at least one client node group 202 and the aggregated value of multiple metric parameters. In this example, the value of L (i.e., the number of score thresholds or the number of client node groups 202) can be 1 or an integer greater than 1.

[0116] In the above AUC calculation, it is necessary to determine the values ​​of each required metric parameter based on the label data of the data sample 102 .

[0117] Alternatively or additionally, in some embodiments, the predetermined performance metric may include an ACC of the predicted result. In this example, for each client node group 202, service node 120 may determine (TP'+TN') and (TP'+FP'+FN'+TN') as aggregate values ​​of multiple metric parameters corresponding to the client node group 202. Service node 120 may then determine the value of ACC based on the aggregate values ​​using the following equation (9).

[0118] ACC=(TP'+TN') / (TP'+FP'+FN'+TN') (9)

[0119] In addition to AUC and ACC, the performance indicators of the machine learning model 130 may also include precision (Precision), which is expressed as Precision = TP' / TP'+FP'. The performance indicators of the machine learning model 130 may also include precision representation, which is the probability of a subset of data samples predicted to be positive samples being labeled as positive samples. The performance indicators of the machine learning model 130 may also include recall (Recall), which is expressed as Recall = TP' / TP'+FN', that is, the probability that a positive sample is predicted. The performance indicators of the machine learning model 130 may also include a PR curve, which has recall as the horizontal axis and precision as the vertical axis. The closer the PR curve is to the upper right corner, the better the performance of the model. The area under the curve is called the AP score (Average Precision Score).

[0120] It should be understood that the performance indicators listed above, such as ROC AUC, are merely exemplary and not restrictive. Examples of performance indicators used in this disclosure include, but are not limited to, ROC AUC, accuracy, error rate, precision, recall, AP score, and the like.

[0121] In the above manner, the service node 120 can determine a predetermined performance indicator based on the perturbation values ​​of multiple metric parameters received from at least one client node group, thereby performing a model performance evaluation. In this way, the client node does not need to expose its local true value label set or its local predicted classification results (i.e., predicted label information). At the same time, the service node can also calculate the value of the performance indicator based on the feedback information of the client node (e.g., the perturbation values ​​of multiple metric parameters). In this way, while determining the performance indicator of the machine learning model, the privacy protection purpose of the local label data of the client node is achieved.

[0122] Figure 5 Flowchart showing a process 500 for model performance evaluation according to some embodiments of the present disclosure. Process 500 may be implemented at client node 110.

[0123] At block 510, the client node 110 determines a plurality of predicted classification results corresponding to the plurality of data samples 102 by comparing the plurality of prediction scores output by the machine learning model 130 for the plurality of data samples 102 with a score threshold. The plurality of predicted classification results respectively indicate that the plurality of data samples 102 are predicted to belong to the first category or the second category. In some embodiments, the process 500 further includes receiving the score threshold from the service node 120. The client node 110 may determine the plurality of predicted classification results corresponding to the plurality of data samples 102 by comparing the plurality of prediction scores output by the machine learning model 130 for the plurality of data samples 102 with the score threshold received from the service node 120.

[0124] At box 520, the client node 110 determines the values ​​of multiple metric parameters related to the predetermined performance indicators of the machine learning model 130 based on the differences between the multiple predicted classification results and the multiple true classification results corresponding to the multiple data samples 102.

[0125] In some embodiments, in order to determine the values ​​of multiple metric parameters, the client node 110 can determine at least one of the following based on the above-mentioned differences: a first number of first-category data samples in multiple data samples, and the predicted classification results and the true classification results corresponding to the first-category data samples both indicate the first category; a second number of second-category data samples in multiple data samples, and the predicted classification results and the true classification results corresponding to the second-category data samples both indicate the second category; a third number of third-category data samples in multiple data samples, and the predicted classification results corresponding to the third-category data samples indicate the first category and the corresponding true classification results indicate the second category; and a fourth number of fourth-category data samples in multiple data samples, and the predicted classification results corresponding to the fourth-category data samples indicate the second category and the corresponding true classification results indicate the first category.

[0126] At block 530, the client node 110 applies a perturbation to the values ​​of the plurality of metric parameters to obtain perturbation values ​​of the plurality of metric parameters. For example, to apply the perturbation to the values ​​of the plurality of metric parameters, the client node 110 may apply the perturbation to at least one of the first number, the second number, the third number, and the fourth number by: determining a sensitivity value associated with the perturbation; determining a random perturbation distribution based on the sensitivity value and the label differential privacy mechanism; and applying the perturbation to the at least one number based on the random perturbation distribution.

[0127] At block 540, client node 110 sends the perturbed values ​​of the plurality of metric parameters to service node 120 for determining a predetermined performance metric at service node 120. For example, the predetermined performance metric includes at least the area under the curve (AUC) of a receiver operating characteristic (ROC) curve.

[0128] Figure 6 A flow chart illustrating a process 600 for model performance evaluation at a service node 120 according to some embodiments of the present disclosure is shown. The process 600 may be implemented at a service node 120 .

[0129] At block 610, the service node 110 receives perturbed values ​​of a plurality of metric parameters associated with a predetermined performance indicator of the machine learning model 130 from the client nodes 110 of at least one group. In some embodiments, the client nodes 110 of each group in the at least one group are different from the client nodes 110 of the other groups. In some embodiments, the process 600 further includes sending at least one score threshold to the client nodes 110 in the respective associated groups.

[0130] In some embodiments, for a given client node 110, the perturbation values ​​of the plurality of metric parameters include at least one of the following: a first perturbation number of data samples of a first class from a plurality of data samples at the given client node 110, the data samples of the first class being labeled as the first category and predicted to be the first category; a second perturbation number of data samples of a second class from a plurality of data samples, the data samples of the second class being labeled as the second category and predicted to be the second category; a third perturbation number of data samples of a third class from a plurality of data samples, the data samples of the third class being labeled as the second category and predicted to be the first category; and a fourth perturbation number of data samples of a fourth class from a plurality of data samples, the data samples of the fourth class being labeled as the first category and predicted to be the second category. The above prediction is based on a comparison of the prediction score output by the machine learning model 130 with a score threshold associated with the group to which the given client node 110 belongs.

[0131] At block 620 , the service node 110 aggregates, for each of the at least one group, the perturbation values ​​of multiple metric parameters of the client nodes 110 in the group by metric parameter to obtain aggregated values ​​of multiple metric parameters respectively corresponding to the at least one group.

[0132] At block 630, service node 110 determines a value for a predetermined performance indicator based on a plurality of score thresholds associated with each of the at least one group and an aggregate value of a plurality of metric parameters corresponding to each of the at least one group. In some embodiments, the at least one group may include a plurality of groups and the at least one score threshold may include a plurality of score thresholds. In such an example, to determine the value for the predetermined performance indicator, service node 120 may determine a receiver operating characteristic (ROC) curve for machine learning model 130 based on the plurality of score thresholds and the aggregate value of the plurality of metric parameters, and determine the area under the ROC curve (AUC).

[0133] Figure 7 1 shows a block diagram of an apparatus 700 for model performance evaluation at a client node according to some embodiments of the present disclosure. The apparatus 700 may be implemented as or included in the client node 110. Each module / component in the apparatus 700 may be implemented by hardware, software, firmware, or any combination thereof.

[0134] As shown, the apparatus 700 includes a classification determination module 710 configured to determine a plurality of predicted classification results corresponding to the plurality of data samples 102 by comparing the plurality of predicted scores output by the machine learning model 130 for the plurality of data samples 102 with a score threshold. The plurality of predicted classification results respectively indicate that the plurality of data samples 102 are predicted to belong to the first category or the second category. In some embodiments, the apparatus 700 further includes a receiving module configured to receive the score threshold from the service node 120. The apparatus 700 can determine a plurality of predicted classification results corresponding to the plurality of data samples 102 by comparing the plurality of predicted scores output by the machine learning model 130 for the plurality of data samples 102 with the score threshold received from the service node 120.

[0135] The device 700 also includes a metric parameter determination module 720, which is configured to determine the values ​​of multiple metric parameters related to predetermined performance indicators of the machine learning model 130 based on the differences between multiple predicted classification results and multiple true classification results corresponding to multiple data samples 102.

[0136] In some embodiments, the metric parameter determination module 720 is configured to determine at least one of the following based on the above-mentioned differences: a first number of data samples of the first category among multiple data samples, the predicted classification results and the true classification results corresponding to the data samples of the first category both indicate the first category; a second number of data samples of the second category among multiple data samples, the predicted classification results and the true classification results corresponding to the data samples of the second category both indicate the second category; a third number of data samples of the third category among multiple data samples, the predicted classification results corresponding to the data samples of the third category indicate the first category and the corresponding true classification results indicate the second category; and a fourth number of data samples of the fourth category among multiple data samples, the predicted classification results corresponding to the data samples of the fourth category indicate the second category and the corresponding true classification results indicate the first category.

[0137] The apparatus 700 further includes a perturbation module 730 configured to apply perturbations to the values ​​of the plurality of metric parameters to obtain perturbation values ​​of the plurality of metric parameters. For example, the perturbation module 730 may be configured to apply perturbations to at least one of the first number, the second number, the third number, and the fourth number by: determining a sensitivity value associated with the perturbation; determining a random perturbation distribution based on the sensitivity value and a label differential privacy mechanism; and applying perturbations to the at least one number based on the random perturbation distribution.

[0138] The apparatus 700 further includes a disturbance value sending module 740 configured to send the disturbance values ​​of the plurality of metric parameters to the service node 120 for determining a predetermined performance indicator at the service node 120. For example, the predetermined performance metric indicator includes at least an area under the curve (AUC) of a receiver operating characteristic (ROC) curve.

[0139] Figure 8 A block diagram of an apparatus 800 for model performance evaluation at a service node according to some embodiments of the present disclosure is shown. The apparatus 800 may be implemented as or included in the service node 120. Each module / component in the apparatus 800 may be implemented by hardware, software, firmware, or any combination thereof.

[0140] As shown, the apparatus 800 includes a perturbation value receiving module 810 configured to receive perturbation values ​​of a plurality of metric parameters related to a predetermined performance indicator of the machine learning model 130 from the client nodes 110 of at least one group. In some embodiments, the client nodes 110 of each group in the at least one group are different from the client nodes 110 of the other groups. In some embodiments, the apparatus 800 may further include a sending module configured to send at least one score threshold to the client nodes 110 in the respective associated groups.

[0141] In some embodiments, for a given client node 110, the perturbation values ​​of the plurality of metric parameters include at least one of the following: a first perturbation number of data samples of a first class from a plurality of data samples at the given client node 110, the data samples of the first class being labeled as the first category and predicted to be the first category; a second perturbation number of data samples of a second class from a plurality of data samples, the data samples of the second class being labeled as the second category and predicted to be the second category; a third perturbation number of data samples of a third class from a plurality of data samples, the data samples of the third class being labeled as the second category and predicted to be the first category; and a fourth perturbation number of data samples of a fourth class from a plurality of data samples, the data samples of the fourth class being labeled as the first category and predicted to be the second category. The above prediction is based on a comparison of the prediction score output by the machine learning model 130 with a score threshold associated with the group to which the given client node 110 belongs.

[0142] The apparatus 800 further includes an aggregation module 820 configured to aggregate, for each of at least one group, disturbance values ​​of multiple metric parameters of the client nodes 110 of the group according to the metric parameters to obtain aggregate values ​​of multiple metric parameters corresponding to the at least one group.

[0143] The apparatus 800 further includes an indicator determination module 830 configured to determine a value of a predetermined performance indicator based on at least one score threshold value associated with each of the at least one group and an aggregate value of a plurality of metric parameters corresponding to each of the at least one group. In some embodiments, the at least one group may include a plurality of groups and the at least one score threshold value includes a plurality of score threshold values. In such embodiments, the indicator determination module 830 includes a receiver operating characteristic (ROC) curve determination module configured to determine the ROC of the machine learning model 130 based on the plurality of score threshold values ​​and the aggregate value of the plurality of metric parameters. The indicator determination module 830 further includes an area under the curve (AUC) determination module configured to determine the AUC of the ROC.

[0144] Figure 9 9 shows a block diagram of a computing device / system 900 capable of implementing one or more embodiments of the present disclosure. It should be understood that Figure 9 The illustrated computing device / system 900 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 9 The computing device / system 900 shown may be used to implement Figure 1 The client node 110 or the service node 120 of FIG.

[0145] like Figure 9 As shown, computing device / system 900 is in the form of a general-purpose computing device. Components of computing device / system 900 may include, but are not limited to, one or more processors or processing units 910, memory 920, storage devices 930, one or more communication units 940, one or more input devices 950, and one or more output devices 960. Processing unit 910 may be a real or virtual processor and is capable of performing various processes according to a program stored in memory 920. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of computing device / system 900.

[0146] The computing device / system 900 typically includes a plurality of computer storage media. Such media can be any available media accessible to the computing device / system 900, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 920 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 930 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk, or any other medium that can be used to store information and / or data (e.g., training data for training) and can be accessed within the computing device / system 900.

[0147] The computing device / system 900 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 9 As shown in FIG, a magnetic disk drive for reading from or writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. Memory 920 may include a computer program product 925 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0148] The communication unit 940 enables communication with other computing devices via a communication medium. Additionally, the functionality of the components of the computing device / system 900 can be implemented as a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the computing device / system 900 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or other network nodes.

[0149] Input device 950 may be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 960 may be one or more output devices, such as a display, speaker, printer, etc. Computing device / system 900 may also communicate with one or more external devices (not shown) via communication unit 940 as needed, such as storage devices, display devices, etc., with one or more devices that allow a user to interact with computing device / system 900, or with any device that allows computing device / system 900 to communicate with one or more other computing devices (e.g., a network card, modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0150] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions or a computer program are stored, wherein the computer-executable instructions or the computer program are executed by a processor to implement the method described above.

[0151] According to an exemplary implementation of the present disclosure, a computer program product is further provided. The computer program product is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions. The computer-executable instructions are executed by a processor to implement the method described above.

[0152] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0153] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0154] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0155] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0156] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, not exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for evaluating model performance, comprising: At the client node, determining a plurality of predicted classification results corresponding to the plurality of data samples by comparing a plurality of prediction scores output by the machine learning model for the plurality of data samples with a score threshold, the plurality of predicted classification results respectively indicating that the plurality of data samples are predicted to belong to the first category or the second category; determining, based on differences between the plurality of predicted classification results and a plurality of true classification results corresponding to the plurality of data samples, values ​​of a plurality of metric parameters associated with predetermined performance indicators of the machine learning model; Applying perturbations to the values ​​of the plurality of measurement parameters to obtain perturbation values ​​of the plurality of measurement parameters; Sending the perturbation values ​​of the multiple metric parameters to a service node, where the perturbation values ​​of the multiple metric parameters are used to perform performance evaluation on the machine learning model at the service node; The score threshold is received from the service node, and the score threshold is associated with a client node of a group to which the client node belongs.

2. The method of claim 1 , wherein determining the values ​​of the plurality of metric parameters comprises: Based on the difference, at least one of the following is determined: a first number of data samples of a first category among the plurality of data samples, wherein both predicted classification results and true classification results corresponding to the data samples of the first category indicate the first category; a second number of data samples of the second category in the plurality of data samples, wherein both the predicted classification results and the true classification results corresponding to the data samples of the second category indicate the second category; a third number of data samples of a third category among the plurality of data samples, the predicted classification results corresponding to the data samples of the third category indicating the first category and the corresponding true classification results indicating the second category; as well as A fourth number of data samples of a fourth category in the plurality of data samples, the predicted classification results corresponding to the data samples of the fourth category indicating the second category and the corresponding true classification results indicating the first category.

3. The method of claim 2 , wherein applying a perturbation to the values ​​of the plurality of metric parameters comprises: For at least one of the first number, the second number, the third number, and the fourth number, a perturbation is applied to the at least one number by: Determine the sensitivity value associated with the disturbance; Determining a random perturbation distribution based on the sensitivity value and the label differential privacy mechanism; and A perturbation is applied to the at least one number based on the random perturbation distribution.

4. The method according to any one of claims 1 to 3, wherein the predetermined performance indicator comprises at least the area under the curve (AUC) of a receiver operating characteristic (ROC) curve.

5. A method for evaluating model performance, comprising: At the service node, perturbation values ​​of a plurality of metric parameters associated with a predetermined performance indicator of the machine learning model are respectively received from at least one group of client nodes; For each group of the at least one group, aggregating the disturbance values ​​of the multiple metric parameters from the client nodes of the group according to the metric parameters to obtain aggregated values ​​of the multiple metric parameters respectively corresponding to the at least one group; as well as determining a value of the predetermined performance indicator based on at least one score threshold value respectively associated with the at least one group and an aggregate value of the plurality of metric parameters respectively corresponding to the at least one group, The method further includes: sending the at least one score threshold to the client nodes in the respective associated groups.

6. The method of claim 5, wherein for a given client node, the perturbed values ​​of the plurality of metric parameters include at least one of the following: a first perturbed number of data samples of a first type among a plurality of data samples at the given client node, the data samples of the first type being labeled as a first class and predicted as the first class; a second perturbed number of data samples of a second category in the plurality of data samples, the data samples of the second category being labeled as a second category and predicted as the second category; a third perturbed number of data samples of a third category in the plurality of data samples, the data samples of the third category being labeled as the second category and predicted as the first category; and a fourth perturbed number of data samples of a fourth category in the plurality of data samples, the data samples of the fourth category being labeled as the first category and predicted as the second category; and The prediction is based on a comparison of a prediction score output by the machine learning model with a score threshold associated with a group to which the given client node belongs.

7. The method of claim 6, wherein the at least one group comprises a plurality of groups and the at least one score threshold comprises a plurality of score thresholds, and wherein determining the value of the predetermined performance indicator comprises: Determining a receiver operating characteristic (ROC) curve of the machine learning model based on the multiple score thresholds and the aggregated values ​​of the multiple metric parameters; as well as The area under the ROC curve, AUC, was determined.

8. The method according to any one of claims 5 to 7, wherein the client nodes of each group in the at least one group are different from the client nodes of other groups.

9. A device for model performance evaluation, comprising: a classification determination module, configured to determine a plurality of predicted classification results corresponding to the plurality of data samples by comparing a plurality of prediction scores output by the machine learning model for the plurality of data samples with a score threshold, the plurality of predicted classification results respectively indicating that the plurality of data samples are predicted to belong to the first category or the second category; a metric parameter determination module configured to determine values ​​of a plurality of metric parameters associated with predetermined performance indicators of the machine learning model based on differences between the plurality of predicted classification results and a plurality of true classification results corresponding to the plurality of data samples; a perturbation module, configured to apply perturbations to the values ​​of the plurality of measurement parameters to obtain perturbation values ​​of the plurality of measurement parameters; as well as a disturbance value sending module, configured to send the disturbance values ​​of the multiple metric parameters to a service node, wherein the disturbance values ​​of the multiple metric parameters are used to perform performance evaluation on the machine learning model at the service node; The score threshold is received from the service node, and the score threshold is associated with a client node in a group to which the client node belongs.

10. The apparatus according to claim 9, wherein the metric parameter determination module is configured to determine at least one of the following based on the difference: a first number of data samples of a first category among the plurality of data samples, wherein both predicted classification results and true classification results corresponding to the data samples of the first category indicate the first category; a second number of data samples of the second category in the plurality of data samples, wherein both the predicted classification results and the true classification results corresponding to the data samples of the second category indicate the second category; a third number of data samples of a third category in the plurality of data samples, the predicted classification results corresponding to the data samples of the third category indicating the first category and the corresponding true classification results indicating the second category; and A fourth number of data samples of a fourth category in the plurality of data samples, the predicted classification results corresponding to the data samples of the fourth category indicating the second category and the corresponding true classification results indicating the first category.

11. The apparatus according to claim 10, wherein the perturbation module is configured to: For at least one of the first number, the second number, the third number, and the fourth number, a perturbation is applied to the at least one number by: Determine the sensitivity value associated with the disturbance; Determining a random perturbation distribution based on the sensitivity value and the label differential privacy mechanism; and A perturbation is applied to the at least one number based on the random perturbation distribution.

12. A device for model performance evaluation, comprising: a disturbance value receiving module configured to receive disturbance values ​​of a plurality of metric parameters related to a predetermined performance indicator of the machine learning model from client nodes of at least one group, respectively; an aggregation module configured to aggregate, for each of the at least one group, the disturbance values ​​of the multiple metric parameters from the client nodes of the group according to the metric parameters to obtain aggregated values ​​of the multiple metric parameters respectively corresponding to the at least one group; as well as an indicator determination module configured to determine a value of the predetermined performance indicator based on at least one score threshold value respectively associated with the at least one group and an aggregate value of the plurality of metric parameters respectively corresponding to the at least one group; The apparatus further includes a sending module configured to send the at least one score threshold to the client nodes in the respective associated groups.

13. The apparatus of claim 12, wherein for a given client node, the perturbed values ​​of the plurality of metric parameters comprise at least one of the following: a first perturbed number of data samples of a first type among a plurality of data samples at the given client node, the data samples of the first type being labeled as a first class and predicted as the first class; a second perturbed number of data samples of a second category in the plurality of data samples, the data samples of the second category being labeled as a second category and predicted as the second category; a third perturbed number of data samples of a third category in the plurality of data samples, the data samples of the third category being labeled as the second category and predicted as the first category; and a fourth perturbed number of data samples of a fourth category in the plurality of data samples, the data samples of the fourth category being labeled as the first category and predicted as the second category; and The prediction is based on a comparison of a prediction score output by the machine learning model with a score threshold associated with a group to which the given client node belongs.

14. The apparatus of claim 13, wherein the at least one group comprises a plurality of groups and the at least one score threshold comprises a plurality of score thresholds, and wherein the indicator determination module comprises: a characteristic curve ROC determination module, configured to determine a receiver operating characteristic curve (ROC) of the machine learning model based on the multiple score thresholds and the aggregated values ​​of the multiple metric parameters; as well as The area under the curve (AUC) determination module is configured to determine the area under the curve (AUC) of the ROC.

15. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the apparatus to perform the method according to any one of claims 1 to 4 or the method according to any one of claims 5 to 8. 16 . A computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method according to claim 1 or the method according to claim 5 .

Images