Method, device, apparatus and medium for model performance evaluation

Abstract

According to an embodiment of the present disclosure, a method, apparatus, device and medium for model performance evaluation are provided. The method includes: at a client node, obtaining multiple prediction scores output by a machine learning model for multiple data samples, the multiple prediction scores respectively indicating the prediction probability that the multiple data samples belong to the first category or the second category; modifying multiple true value labels based on a random response mechanism to obtain multiple protected labels, the multiple true value labels respectively annotate that the multiple data samples belong to the first category or the second category; based on the multiple protected labels and the multiple prediction scores, determining error measurement information related to a predetermined performance indicator of the machine learning model; and sending the error measurement information to a service node. Thus, while achieving model performance evaluation, the purpose of protecting the privacy of the local label data of the client node is achieved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly to methods, devices, apparatuses, and computer-readable storage media for model performance evaluation. Background Art

[0002] Machine learning has been widely used, and its performance usually improves with the increase of data volume. Ideally, it can be considered that high-quality data samples and sufficient labeled data can be collected centrally for training machine learning models. However, in many real-world scenarios, there is the so-called data island problem, that is, data is usually dispersed and isolated and stored on different entities (for example, enterprises, user terminals). As data privacy protection issues are increasingly valued, it is difficult to further improve the current centralized machine learning system. Therefore, federated learning has emerged. Federated learning can achieve performance consistent with traditional machine learning algorithms in an encrypted environment when data leaves the local node.

[0003] In federated learning, it is expected to better protect data privacy, including the privacy of label data corresponding to data samples. Summary of the invention

[0004] According to an example embodiment of the present disclosure, a scheme for model performance evaluation is provided.

[0005] In a first aspect of the present disclosure, a method for model performance evaluation is provided. The method includes obtaining, at a client node, a plurality of prediction scores output by a machine learning model for a plurality of data samples, the plurality of prediction scores respectively indicating the prediction probabilities that the plurality of data samples belong to a first category or a second category; modifying a plurality of true value labels based on a random response mechanism to obtain a plurality of protected labels, the plurality of true value labels respectively marking that the plurality of data samples belong to the first category or to the second category; determining error metric information related to a predetermined performance indicator of the machine learning model based on the plurality of protected labels and the plurality of prediction scores; and sending the error metric information to a service node.

[0006] In a second aspect of the present disclosure, a method for model performance evaluation is provided. The method includes receiving, at a service node, error metric information related to a predetermined performance indicator of a machine learning model from a plurality of client nodes, respectively, the error metric information being determined by the corresponding client nodes based on a plurality of protected tags, respectively, the plurality of protected tags being generated by applying a random response mechanism to a plurality of true value tags; determining an error value of the predetermined performance indicator based on the error metric information; and determining a corrected value of the predetermined performance indicator by correcting the error value.

[0007] In a third aspect of the present disclosure, a device for model performance evaluation is provided. The device includes a score acquisition module, configured to obtain multiple prediction scores output by a machine learning model for multiple data samples, the multiple prediction scores respectively indicating the prediction probability that the multiple data samples belong to the first category or the second category; a label modification module, configured to modify multiple true value labels based on a random response mechanism to obtain multiple protected labels, the multiple true value labels respectively mark that the multiple data samples belong to the first category or the second category; an information determination module, configured to determine error metric information related to a predetermined performance indicator of the machine learning model based on the multiple protected labels and the multiple prediction scores; and an information sending module, configured to send the error metric information to a service node.

[0008] In a fourth aspect of the present disclosure, a device for model performance evaluation is provided. The device includes: an information receiving module configured to receive error measurement information related to a predetermined performance indicator of a machine learning model from a plurality of client nodes, respectively, the error measurement information being determined by the corresponding client nodes based on their respective plurality of protected tags, the plurality of protected tags being generated by applying a random response mechanism to a plurality of true value tags; an indicator determination module configured to determine an error value of a predetermined performance indicator based on the error measurement information; and an indicator correction module configured to determine a correction value of the predetermined performance indicator by correcting the error value.

[0009] In a fifth aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0010] In a sixth aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the second aspect.

[0011] In a seventh aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and the computer program is executed by a processor to implement the method of the first aspect.

[0012] In an eighth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and the computer program is executed by a processor to implement the method of the second aspect.

[0013] It should be understood that the contents described in the summary of the present invention are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0015] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be applied;

[0016] Figure 2 A flow chart showing a signaling flow for model performance evaluation according to some embodiments of the present disclosure;

[0017] Figure 3 A schematic diagram showing an example of applying a random response mechanism to a true value label according to some embodiments of the present disclosure;

[0018] Figure 4 A flowchart illustrating a process of type performance evaluation at a client node according to some embodiments of the present disclosure;

[0019] Figure 5 A flowchart illustrating a process of model performance evaluation at a service node according to some embodiments of the present disclosure;

[0020] Figure 6 A block diagram of an apparatus for model performance evaluation at a client node according to some embodiments of the present disclosure is shown;

[0021] Figure 7 A block diagram showing an apparatus for model performance evaluation at a service node according to some embodiments of the present disclosure; and

[0022] Figure 8 A block diagram of a computing device / system capable of implementing one or more embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0023] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0024] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0025] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0026] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0027] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0028] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information is sent to the user in a manner such as a pop-up window, in which the prompt information can be presented in text form. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0029] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0030] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. A neural network model is an example of a model based on deep learning. In this article, "model" may also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.

[0031] A "neural network" is a machine learning network based on deep learning. A neural network is capable of processing inputs and providing corresponding outputs, and typically includes an input layer and an output layer and one or more hidden layers between the input layer and the output layer. Neural networks used in deep learning applications typically include many hidden layers, thereby increasing the depth of the network. The layers of a neural network are connected in sequence so that the output of the previous layer is provided as input to the next layer, where the input layer receives the input of the neural network and the output of the output layer serves as the final output of the neural network. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each of which processes input from the previous layer.

[0032] Generally, machine learning can be roughly divided into three stages, namely the training stage, the testing stage, and the application stage (also called the inference stage). In the training stage, a given model can be trained using a large amount of training data, and the parameter values ​​are continuously updated iteratively until the model can obtain consistent inferences that meet the expected goals from the training data. Through training, the model can be considered to be able to learn the association from input to output (also called the mapping of input to output) from the training data. The parameter values ​​of the trained model are determined. In the testing stage, the test input is applied to the trained model to test whether the model can provide the correct output, thereby determining the performance of the model. In the application stage, the model can be used to process the actual input based on the parameter values ​​obtained from the training to determine the corresponding output.

[0033] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. The environment 100 relates to a federated learning environment, which includes N client nodes 110-1...110-k,...110-N (where N is an integer greater than 1, k=1, 2,...N) and a service node 120. The client nodes 110-1...110-k,...110-N can respectively maintain their own local data sets 112-1...112-k,...112-N. For ease of discussion, the client nodes 110-1...110-k,...110-N may be collectively or individually referred to as client nodes 110, and the local data sets 112-1...112-k,...112-N may be collectively or individually referred to as local data sets 112.

[0034] In some embodiments, client node 110 and / or service node 120 can be implemented at terminal device or server.Terminal device can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phone, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, media computer, multimedia tablet, personal communication system (PCS) equipment, personal navigation device, personal digital assistant (PDA), audio / video player, digital camera / camcorder, positioning device, television receiver, radio broadcast receiver, electronic book device, game device or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof.In some embodiments, terminal device can also support any type of interface for user (such as "wearable" circuit, etc.).Server is various types of computing system / server that can provide computing power, including but not limited to computing equipment in mainframe, edge computing node, cloud environment, etc.

[0035] In federated learning, a client node refers to a node that provides part of the training data for a machine learning model. A client node may also be called a client, terminal node, terminal device, user device, etc. In federated learning, a service node refers to a node that aggregates the training results at the client nodes.

[0036] exist Figure 1 In the example, it is assumed that N client nodes 110 jointly participate in the training of the machine learning model 130, and aggregate the intermediate results of the training to the service node 120, so that the service node 120 updates the parameter set of the machine learning model 130. The full set of local data of these client nodes 110 constitutes a complete training data set of the machine learning model 130. Therefore, according to the mechanism of federated learning, the service node 120 will generate a global machine learning model 130.

[0037] For the machine learning model 130, the local dataset 112 at the client node 110 may include data samples and true value labels. Figure 1Specifically shown is a local data set 112-k at a certain client node 110-k, which includes a data sample set and a true value label set. The data sample set includes multiple (M) data samples 102-1, 102-i, ... 102-M (collectively or individually referred to as data samples 102), and the true value label set includes corresponding multiple (M) ground-truth labels 105-1, 105-i, ... 105-M (collectively or individually referred to as true value labels 105). Wherein, M is an integer greater than 1, i = 1, 2, ... M. Each data sample 102 can be annotated with a corresponding true value label 105. The data sample 102 can correspond to the input of the machine learning model 130, and the true value label 105 indicates the true output of the corresponding data sample 102. The true value label is an important part of supervised machine learning.

[0038] In an embodiment of the present disclosure, the machine learning model 130 can be constructed based on various machine learning or deep learning model architectures, and can be configured to implement various prediction tasks, such as various classification tasks, recommendation tasks, etc. The machine learning model 130 may also be referred to as a prediction model, a recommendation model, a classification model, and the like.

[0039] The data sample 102 may include input information related to a specific task of the machine learning model 130, and the truth label 105 is related to the expected output of the task. As an example, in a binary classification task, the machine learning model 130 may be configured to predict whether the input data sample belongs to the first category or the second category, and the truth label is used to mark whether the data sample actually belongs to the first category or the second category. Many practical applications can be classified as such binary classification tasks, such as whether the recommended item is converted (e.g., clicked, purchased, registered, or other demand behaviors) in a recommendation task, etc.

[0040] It should be understood that Figure 1 Only an example federated learning environment is shown. Depending on the federated learning algorithm and actual application needs, the environment may also be different. For example, although shown as a separate node, in some applications, the service node 120 may serve as a client node in addition to serving as a central node to provide some data for model training, model performance evaluation, etc. The embodiments of the present disclosure are not limited in this respect.

[0041] During the training phase of the machine learning model 130, some mechanisms have been developed to protect the local data of each client node 110 from being leaked. For example, during the model training process, the client node 110 does not need to disclose local data samples or label data, but instead sends the gradient data calculated based on the local training data to the service node 120 for the service node 120 to update the parameter set of the machine learning model 130.

[0042] In some cases, it is also desirable to globally evaluate the performance of the trained machine learning model. The performance of a machine learning model can be measured by one or more performance indicators. Different performance indicators can measure the difference between the predicted output given by the machine learning model for the data sample set and the true output indicated by the true value label set from different perspectives. Generally, if the difference between the predicted output given by the machine learning model and the true output is small, it means that the performance of the machine learning model is good. It can be seen that it is usually necessary to determine the performance indicators of the machine learning model based on the true value label set of the data sample.

[0043] As the data supervision system continues to strengthen, the requirements for data privacy protection are becoming increasingly stringent, including the protection of the true value labels of data samples to prevent them from being leaked. For example, for the data owner in the recommendation task, the user's actual conversion behavior for the recommended item involves user privacy and is sensitive information that needs to be protected.

[0044] Therefore, it is a challenging task to determine the performance indicators of the machine learning model while protecting the local label data of the client node from being leaked. Currently, there is no very effective solution to this problem.

[0045] According to an embodiment of the present disclosure, a model performance evaluation scheme is provided, which is capable of protecting the local label data of the client node. Specifically, at the client node, the true value label set corresponding to the data sample set is modified by applying a randomized response (RR) mechanism to obtain a protected label set. The client node determines the measurement information related to the performance indicators of the machine learning model based on the protected label set and the predicted score output by the machine learning model for the data sample set. Here, since the label set is a modified protected label set, the determined measurement information is not accurate measurement information, which is called "error measurement information". The client node sends the error measurement information to the service node.

[0046] At the service node, the service node receives respective error metric information from the plurality of client nodes, and determines an error value of the performance indicator based on the error metric information. The service node further corrects the error value to obtain a corrected value of the performance indicator.

[0047] According to the embodiments of the present disclosure, each client node does not need to expose the local true value label set, and the service node can also calculate the value of the performance indicator based on the feedback information of the client node. In this way, while realizing the model performance evaluation, the privacy protection purpose of the local label data of the client node is achieved.

[0048] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0049] Figure 2 200 for model performance evaluation according to some embodiments of the present disclosure. Figure 1 The signaling flow 200 involves a client node 110 and a service node 120.

[0050] In an embodiment of the present disclosure, it is assumed that the performance of the machine learning model 130 is to be evaluated. In some embodiments, the machine learning model 130 to be evaluated may be a global machine learning model determined based on a training process of federated learning, for example, the client node 110 and the service node 120 participate in the training process of the machine learning model 130. In some embodiments, the machine learning model 130 may also be a model obtained in any other manner, and the client node 110 and the service node 120 may not participate in the training process of the machine learning model 130. The scope of the present disclosure is not limited in this respect.

[0051] In some embodiments, as shown in the signaling flow 200, the service node 120 sends 205 the machine learning model 130 to the N client nodes 110. After receiving 210 the machine learning model 130, each client node 110 may perform a subsequent evaluation process based on the machine learning model 130. In some embodiments, the machine learning model 130 to be evaluated may also be provided to the client node 110 in any other suitable manner.

[0052] In the embodiments of the present disclosure, the operations of a client node are described from the perspective of a single client node.

[0053] In the process of performing the model performance evaluation, the client node 110 obtains 215 a plurality of prediction scores output by the machine learning model 130 for the plurality of data samples 102. In some embodiments, the client node 110 may apply each data sample 102 to the machine learning model 130 as an input to the model, and obtain the prediction score output by the machine learning model 130. For example, assuming that the data sample set of the client node 110-k is X k , the machine learning model 130 is represented as f(), then the set of prediction scores for the data sample set can be represented as s k =f(X k ), where k = 1, 2, ... N.

[0054] In the embodiments of the present disclosure, particular attention is paid to the performance indicators of the machine learning model in implementing the binary classification task. Each prediction score can indicate the predicted probability that the corresponding data sample 102 belongs to the first category or the second category. The two categories can be configured according to the actual task needs.

[0055] The value range of the prediction score output by the machine learning model 130 can be set arbitrarily. For example, the prediction score can be a value in a certain continuous value interval (for example, a value between 0 and 1), or it can be a value in a plurality of discrete values ​​(for example, it can be one of the discrete values ​​of 0, 1, 2, 3, 4, 5, etc.). In some examples, a higher prediction score can indicate that the predicted probability that the data sample 102 belongs to the first category is greater, and the predicted probability that it belongs to the second category is smaller. Of course, the opposite setting is also possible, for example, a higher prediction score can indicate that the predicted probability that the data sample 102 belongs to the second category is greater, and the predicted probability that it belongs to the first category is smaller.

[0056] The client node 110 also modifies 220 the plurality of true value labels 105 (also referred to as real value labels) corresponding to each of the plurality of data samples 102 based on a random response mechanism to obtain a plurality of protected labels.

[0057] It should be understood that although the acquisition of the prediction score at 215 and the random response mechanism applied to the true value label at 220 are described in sequence, these two operations can be performed in any order without limitation.

[0058] The true value label 105 is used to mark whether the corresponding data sample 102 belongs to the first category or the second category. In the following, for the convenience of discussion, the data samples belonging to the first category are sometimes referred to as positive samples, positive examples or positive class samples, and the data samples belonging to the second category are sometimes referred to as negative samples, negative examples or negative class samples. In some embodiments, each true value label 105 can have one of two values, which are used to indicate the first category or the second category respectively. In some embodiments below, for the convenience of discussion, the value of the true value label 105 corresponding to the first category can be set to "1", which indicates that the data sample belongs to the first category and is a positive sample. In addition, the value of the true value label 105 corresponding to the second category can be set to "0", which indicates that the data sample belongs to the second category and is a negative sample.

[0059] In an embodiment of the present disclosure, in order to achieve privacy protection for the true value label while determining the performance indicators of the machine learning model 130, the true value label is converted into a protected label through a random response mechanism. Figure 3 FIG. 2 shows an example of a protected label obtained after applying a random response mechanism to the true value label 105 according to some embodiments of the present disclosure. Figure 3 As shown, after the random response mechanism is applied, the K true value labels 105 corresponding to the K data samples 102 will correspond to the protected labels 305-1, ... 305-i, ... 305-K (collectively or individually referred to as protected labels 305).

[0060] The random response mechanism is a differential privacy (DP) mechanism. To better understand the embodiments of the present disclosure, differential privacy and the random response mechanism will be briefly introduced below.

[0061] Assume ∈, δ is a real number greater than or equal to 0, that is, and It is a random mechanism (random algorithm). The so-called random mechanism means that for a specific input, the output of the mechanism is not a fixed value, but obeys a certain distribution. A random mechanism can be considered if the following conditions are met With (∈, δ)-differential privacy: For any two adjacent training datasets D, D′, and for For any subset S of the possible outputs, there exists:

[0062]

[0063] In addition, if δ = 0, it can also be considered that the random mechanism With ∈-differential privacy (∈-DP). In the differential privacy mechanism, for a random mechanism with (∈, δ)-differential privacy or ∈-differential privacy It is expected that the distributions of the two outputs obtained after applying them to two adjacent data sets are indistinguishable. In this way, observers can hardly detect small changes in the input data set of the algorithm by observing the output results, thus achieving the purpose of privacy protection. If the probability of obtaining a specific output S is similar when applied to any adjacent data set, then it will be considered that the algorithm is unlikely to achieve differential privacy.

[0064] In the embodiments of this article, we focus on the differential privacy of the labels of data samples, and the labels indicate the binary classification results. Therefore, following the setting of differential privacy, we can define label differential privacy. Specifically, assuming ∈, δ is a real number greater than or equal to 0, that is, and It is a random mechanism (random algorithm). If the following conditions are met, it can be considered a random mechanism With (∈, δ)-label differential privacy: for any two adjacent training datasets D, D′, their difference lies only in the label of a single data sample, and for For any subset S of the possible outputs, there exists:

[0065]

[0066] In addition, if δ = 0, it can also be considered that the random mechanism It has ∈-differential privacy (∈-DP). That is, it is expected that after changing the label of the data sample, The distribution of the output results is still small, making it difficult for observers to detect the change of labels.

[0067] The random response mechanism is a random mechanism applied for the purpose of differential privacy protection. The random response mechanism is positioned as follows: Assume ∈ is a parameter and y∈[0,1] is the known value of the true value label in the random response mechanism. If for the value y of the true value label, the random response mechanism derives a random value from the following probability distribution

[0068]

[0069] That is, after applying the random response mechanism, the random value There is a certain probability that it is equal to y, and there is also a certain probability that it is not equal to y. The above random response mechanism is considered to have label differential privacy of δ = 0 ((∈, 0)-label differential privacy) because:

[0070]

[0071] That is, the random response mechanism will satisfy ∈-differential privacy.

[0072] Differential privacy and random response mechanisms have been discussed above. When applied to the modification of multiple truth value labels 105 at the client node 110, the values ​​of multiple truth value labels 105 will be randomly changed according to a certain probability distribution. This is also equivalent to adding noise or interference to the set of multiple truth value labels 105. Therefore, the protected label 305 may sometimes also be referred to as a noise label or an interference label.

[0073] Assume that the true value label 105 of the i-th data sample 102 at the client node 110-k is represented as The protected tag 305 is represented as After applying the random response mechanism, from the results, the values ​​of some truth labels 105 may be changed (i.e., ), some truth labels 105 may remain unchanged (i.e., ). Where k = 1, 2, ..., N, i = 1, 2, ..., |X k |, and |X k | is the number of data samples of the client node 110 - k .

[0074] Since the true value label 105 in the binary classification problem is selected from two values, the change of the true value label 105 can be considered as reversing the value of the true value label 105. For example, if the true value label 105 The value is 1. After inversion, the protected tag 305 The value of is 0.

[0075] Through the random response mechanism, since the value of the true value label 105 is randomly changed, the true value label 105 cannot be derived from the protected label 305.

[0076] Continue to return to reference Figure 2 , after obtaining the plurality of prediction scores and the plurality of protected labels, the client node 110 determines 225 metric information related to the predetermined performance indicator of the machine learning model 130. As mentioned above, based on the modified set of protected labels, the metric information determined here is not accurate metric information, and is referred to as "error metric information".

[0077] In an embodiment of the present disclosure, individual client nodes 110 determine metric information related to the performance indicators of the model based on the local data set (data samples and true value labels). The metric information of multiple client nodes 110 can be aggregated to the service node 120. In this way, the performance of the machine learning model 130 is evaluated based on the complete data set of multiple client nodes.

[0078] The type of error metric information provided by the client node may depend on the performance metric to be calculated, and on whether the client node 110 is to provide the protected tag 305 to the serving node.

[0079] In the following, some example performance indicators of the machine learning model 130 for implementing a binary classification task are first introduced, and then how the client node 110 feeds back error metric information to the service node is discussed in detail.

[0080] The prediction score given by the machine learning model 130 for a data sample is usually compared with a certain score threshold, and the data sample is predicted to belong to the first category or the second category based on the comparison result. The prediction of the machine learning model 130 for implementing a binary classification task may have four results.

[0081] Specifically, for a certain data sample 102, assuming that the true value label 105 indicates that it belongs to the first category (positive sample), and the machine learning model 130 also predicts that it is a positive sample, then the data sample is considered to be a true sample (True Positive, TP). If the true value label 105 indicates that it belongs to the first category (positive sample), and the machine learning model 130 predicts that it is a negative sample, then the data sample is considered to be a false negative sample (False Negative, FN). If the true value label 105 indicates that it belongs to the second category (negative sample), but the machine learning model 130 also predicts that it is a negative sample, then the data sample is considered to be a true negative (True Negative, TN). If the true value label 105 indicates that it belongs to the second category (negative sample), but the machine learning model 130 predicts that it is a positive sample, then the data sample is considered to be a false positive (False Positive, FP). These four results can be indicated by the confusion matrix in Table 1 below.

[0082] Table 1

[0083]

[0084] When measuring the performance of the machine learning model 130, it is desirable to be able to calculate performance indicators based on the prediction results of the entire set of data samples from multiple client nodes 110 and the entire set of true value labels.

[0085] In some embodiments, the performance metric of the machine learning model 130 may include the area under the curve (AUC) of the receiver operating characteristic curve (ROC).

[0086] The ROC curve is a curve drawn on the coordinate axis based on different classification methods (setting different score thresholds), with the false positive sample rate (FPR) as the X-axis and the true positive sample rate (TPR) as the Y-axis. FPR can be defined as: the ratio of data samples that are actually negative examples that are incorrectly judged as positive by the model, expressed as FPR = FP / (FP+TN), where FP and TN represent the number of FP and TN counted in the entire data sample set. TPR: The ratio of samples that are actually positive that are correctly judged as positive, expressed as TPR = TP / (TP+FN). According to each possible score threshold, the coordinate points of multiple (FPR, TPR) pairs can be calculated, and these points are connected into a line to form the ROC curve of a specific model.

[0087] From the definition, AUC refers to the area under the ROC curve. When calculating AUC, one possible way is to calculate AUC by using an approximate algorithm to calculate the area under the ROC curve based on the definition of AUC.

[0088] In some embodiments, AUC can also be determined from a probabilistic perspective. AUC can be considered as: randomly selecting a positive sample and a negative sample, the probability that the machine learning model gives a higher prediction score to the positive sample than to the negative sample. That is to say, in the data sample set, the positive and negative samples are combined in pairs to form positive and negative sample pairs, where the prediction score of the positive sample is greater than the prediction score of the negative sample. If the model can output a higher prediction score for more positive samples than for negative samples, it can be considered that the AUC is higher and the model performs better. The value range of AUC is between 0.5 and 1. The closer the AUC is to 1, the better the performance of the model.

[0089] In the above AUC calculations, it is necessary to determine the values ​​of some metric parameters based on the label data of the data samples.

[0090] In addition to AUC, the performance indicators of the machine learning model 130 may also include precision (Precision), which is expressed as Precision = TP / TP + FP. Precision indicates the probability of being labeled as a positive sample in a subset of data samples predicted as positive samples. The performance indicators of the machine learning model 130 may also include recall (Recall), which is expressed as Recall = TP / TP + FN, that is, the probability of a positive sample being predicted. The performance indicators of the machine learning model 130 may also include a PR curve, which has recall as the horizontal axis and precision as the vertical axis. The closer the PR curve is to the upper right corner, the better the performance of the model. The area under the curve is called the AP score (Average Precision Score).

[0091] In the following, the determination of AUC will be mainly discussed as an example.

[0092] Continue to refer Figure 2 After determining the error metric information, the client node 110 sends 230 the determined error metric information to the service node 120 .

[0093] As mentioned above, the client node 110 may choose to send the plurality of protected tags 305 as part of the error metric information to the service node, or may choose not to send the protected tags 305 but continue to calculate the value of further metric parameters based on this.

[0094] In some embodiments where the protected tags 305 are directly sent, the client node 110 may directly determine the plurality of prediction scores and the plurality of protected tags 305 as error metric information, and send the information to the service node 120. Figure 2As shown, in the error metric information transmission method 236, the client node 110 sends 240 a plurality of prediction scores and a plurality of protected labels to the service node 120. Thus, the service node 120 may receive 242 the prediction scores and protected labels to the client node 110. In these embodiments, for each data sample 102, the corresponding prediction score and protected label may be sent to the service node 120 in pairs.

[0095] Figure 2 Another way of sending the error metric information 238 is also shown. In this way, the client node 110 can determine the plurality of prediction scores as a first part of the error metric information and send 244 this part of the information to the service node 120.

[0096] In some embodiments, before sending the prediction scores to the service node 120, the client node 110 may randomly adjust the order of the multiple prediction scores and send the multiple prediction scores to the service node in the adjusted order. By randomly adjusting the order, it is possible to avoid that in some special cases, after the multiple data samples 102 are sequentially input into the model at the client node, the output prediction scores have a certain order, such as from large to small or from small to large, which may cause certain information leakage. Random order adjustment can further strengthen data privacy protection.

[0097] After receiving 246 the prediction scores, the service node 120 sorts 248 the prediction score sets from the plurality of client nodes 110 to obtain a sort result of the prediction score from each client node 110 in the prediction score set.

[0098] In some embodiments, the service node 120 may sort the prediction score set in ascending order and assign each prediction score (prediction score of the i-th data sample of client node 110-k) assigned ranking value In some embodiments, the ranking value Can indicate the prediction score The number of other prediction scores that are exceeded in the set of prediction scores. For example, in ascending order, the lowest prediction score is assigned a ranking value of 0, indicating that it does not exceed (is greater than) any other prediction score; the next prediction score is assigned a ranking value of 1, indicating that it is greater than 1 prediction score in the set, and so on. Such a ranking value assignment is beneficial to subsequent calculations.

[0099] In the manner 238, for the client node 110 that receives the prediction score, the service node 120 sends 250 the ranking result of its multiple prediction scores in the overall prediction score set to the corresponding client node 110. After receiving 252 the ranking result, the client node 110 can determine 254 the second part of the error metric information based on the local multiple protected tags 305 and the ranking results of each of the multiple prediction scores. The second part of the error metric information refers to the value of the metric parameter required to calculate the specific performance metric of the machine learning model 130 in addition to the prediction score.

[0100] In some embodiments where AUC is determined as a performance metric, the client node 110 may determine the number of first-class protected tags (referred to as a "first number") among the plurality of protected tags 305, where the first-class protected tags 305 indicate that the corresponding data sample 102 belongs to the first category, for example, indicating that the data sample 102 is a positive sample. In addition, the client node 110 may also determine the number of second-class protected tags (referred to as a "second number") among the plurality of protected tags 305, where the second-class protected tags indicate that the corresponding data sample belongs to the second category, for example, indicating that the data sample is a negative sample. At the client node 110-k, the determination of the first number and the second number may be expressed as follows:

[0101]

[0102]

[0103] where |X k | represents the number of data samples of the client node 110 - k ; represents the value of the protected label corresponding to the i-th data sample; localP k represents the number of first-class protected labels (labels indicating positive samples) among the protected labels at the client node 110 - k , localP k represents the number of second-category protected tags (tags indicating positive samples) among the protected tags at the client node 110 - k .

[0104] In the above equations (5) and (6), it is assumed that for positive samples, The value of is 1. For negative samples, The value of is 0. The sum of can be used to count the number of positive samples indicated by the protected label. The number of negative samples indicated by the protected label can be counted by adding the positive and negative samples of the protected label. In other examples, if the protected label uses other values ​​to indicate positive and negative samples, localP can also be counted in other ways. kand localP k , this article does not limit this. localP k and localP k The values ​​(error values) of the two metric parameters in the error metric information at the client node 110 - k may be determined.

[0105] In some embodiments, the client node 110 may also determine the number of prediction scores (referred to as a third number) that the prediction scores of the data samples corresponding to the first type of protected labels (i.e., positive samples) exceed in the prediction score set based on the ranking results of each of the multiple prediction scores. This number may indicate the number of sample pairs in the set of data samples of the client node 110 where the ranking of the positive samples is higher than the ranking of the remaining samples (in the case of ascending sorting). In some embodiments, at the client node 110-k, the third number may be determined by:

[0106]

[0107] where localSum k Indicates the third number, represents the value of the protected label corresponding to the i-th data sample, Represents the ranking value of the prediction score corresponding to the i-th data sample. As mentioned above, the ranking value Can be set to indicate the prediction score The number of other prediction scores that are exceeded in the prediction score set. In the above formula (7), it is also assumed that for positive samples, The value of is 1. For negative samples, The value of is 0. By adding the localSum, we can determine the number of samples whose prediction scores rank higher than the prediction scores of the remaining samples (also the number of such prediction scores). k The value (error value) of another metric parameter in the error metric information at the client node 110 - k may be determined.

[0108] localP k 、localN k and localSum k These are all metric parameters that need to be determined in the example calculation method of AUC. The client node 110 may send 256 the values ​​of these three metric parameters to the service node 120 as the second part of the error metric information. After receiving 258 the second part of the error metric information, the service node 120 may perform subsequent operations accordingly.

[0109] In some embodiments, different client nodes 110 may select the method 236 or the method 238 to send respective error metric information to the service node 120 .

[0110] Regardless of whether the protected tag 305 leaves the client node 110, the true value tag can obtain privacy protection. This is because the random response mechanism is immune to post-processing. In other words, after applying the random response mechanism, no matter how the protected tag and its related statistical information are subsequently processed, that is, regardless of whether the protected tag data is sent from the client node, the differential privacy protection capability will not be eliminated.

[0111] After receiving 235 the error metric information sent to each client node 110, the service node 120 determines 260 the value of the performance indicator of the machine learning model 130 based on the error metric information from the plurality of client nodes 110. Here, since the error metric information is used, the determined value of the performance indicator is also referred to as an error value.

[0112] The calculation of the performance indicator based on the measurement information depends on the measurement information obtained and the type of performance indicator to be determined. For the ACU, different algorithms can also be used for flexible determination.

[0113] In some embodiments, if the service node 120 receives the values ​​of the metric parameters localP sent by the method 238 from the plurality of client nodes 110, k 、localN k and localSum k , then the service node 120 can aggregate the values ​​of these metric parameters of multiple client nodes 110 by parameters to obtain the aggregated value (global value) of each metric parameter, as follows:

[0114]

[0115]

[0116]

[0117] in represents the total number of first-category protected labels (labels indicating positive samples) among all protected labels of the plurality of client nodes 110 (referred to as “first total number”), represents the total number of second-class protected labels (labels indicating negative samples) in all protected labels of the plurality of client nodes 110 (referred to as the “second total number”), and globalSum represents the third total number of predicted scores that the predicted scores of data samples corresponding to the first-class protected labels exceed in the predicted score set. Since the statistics are all performed based on the protected labels, There may be errors between the values ​​​​calculated based on the true value labels of the client nodes and globalSum.

[0118] In some embodiments, if the service node 120 receives error metric information from multiple client nodes 110 that is a prediction score and a protected label, such as receiving such error metric information through the method 236, then the service node 120 can aggregate the prediction scores and protected labels of these client nodes 110 and directly calculate the predicted scores and protected labels in a similar manner as discussed above for the client nodes. and globalSum.

[0119] In some embodiments, if the error metric information received by the service node 120 from a certain client node 110 or a certain portion of the client nodes 110 is a prediction score and a protected label, for example, such error metric information is received through the method 236, then the service node 120 calculates the localP corresponding to each client node in a similar manner as discussed above for the client node. k 、localN k and localSum k The service node 120 may also collect the prediction scores and protected labels of some client nodes 110, and count the number of first-class protected labels, the number of second-class protected labels, and the number of prediction scores in the prediction score set that the prediction scores of data samples corresponding to the first-class protected labels exceed according to the similar method discussed above for the client nodes. Then, the service node 120 compares the statistical information with the localP received directly from the remaining client nodes. k 、localN k and localSum k Aggregate to determine and globalSum.

[0120] In some embodiments, based on and globalSum, the service node 120 can calculate the value of AUC in the following manner (the calculated value here is the error value, expressed as AUC_corr):

[0121]

[0122] In some embodiments, if the service node 120 receives prediction scores and protected labels from multiple client nodes 110 through method 236, the service node 120 may also calculate the AUC in other ways. Specifically, the service node 120 may summarize the received prediction scores and protected labels. The service node 120 may determine the number of positive samples indicated by the protected labels and the number of negative samples indicated by the protected labels in the protected label set. In addition, the service node 120 may determine, based on the prediction score set, the number of positive samples whose prediction scores are greater than the prediction scores of negative samples in all data samples. The service node 120 may further calculate the value of the AUC (i.e., the error value) based on these three numbers.

[0123] Assume that based on the number of protected labels or prediction scores, it can be determined that the total number of data samples at N client nodes 110 is L, and the number of positive samples indicated by protected labels is m, and the number of negative samples is n. In addition, the prediction score corresponding to each data sample is s i , i∈[1,L]. By traversing the pairwise combinations of positive and negative samples, m*n sample pairs P can be formed i , i∈[1,m*n], then the AUC can be determined as follows:

[0124] Among them (11)

[0125]

[0126] Some example calculation methods for AUC are discussed above. If appropriate, AUC can also be determined from a probability and statistical perspective based on other methods.

[0127] In some embodiments, in addition to AUC, other performance indicators of the machine learning model 130 may also be evaluated, as long as such performance indicators can be determined from multiple prediction scores and multiple protected labels. The embodiments of the present disclosure are not limited in this respect.

[0128] In order to obtain a more accurate value of the performance indicator, on the basis of the error value of the performance indicator, the service node 120 determines 265 a corrected value of the predetermined performance indicator by correcting the error value.

[0129] In some embodiments, a mapping relationship between an error value and a correction value of a performance indicator may be determined, and the error value may be corrected based on the mapping relationship.

[0130] In some embodiments, for AUC, the mapping relationship between its error value and the correction value may be determined based on a first total number of first-class protected tags and a second total number of second-class protected tags in the protected tag set involved in the N client nodes 110. As an example, the mapping relationship between the error value (AUC_corr) of AUC and the correction value (expressed as AUC_real) may be expressed as follows:

[0131] Among them (12)

[0132]

[0133]

[0134] Where π = P(Y = 1), refers to the proportion of positive samples in the data sample set indicated by the true value label, ρ + and ρ - Indicates the rate of change of the true value labels indicating positive and negative samples in the application of the random response mechanism, respectively.

[0135] For π, since the service node 120 does not know the true value label, the proportion of positive samples in the data sample set indicated by the true value label can be estimated through the protected label. Assume that M and N are the number of positive samples and negative samples in the data sample set indicated by the true value label, is a first total number of first class protected tags and a second total number of second class protected tags determined from the error metric information provided by the client node 110. That is, the total number of samples or labels remains unchanged. In addition, it can also be determined From these two equations, we can obtain:

[0136]

[0137]

[0138] Accordingly, it can be determined that

[0139] Therefore, according to the above formula (12), when π and ρ are known, + and ρ - In this case, AUC_real can be calculated from AUC_corr.

[0140] It can be understood that after correcting the error value of AUC, there may still be some errors between the calculated AUC_real and the AUC calculated based on the true value label. However, according to the results of repeated experiments by the inventors, it can be determined that such errors are small and within the allowable range. In fact, strictly speaking, even with true value labels, many algorithms for calculating AUC approximate the true value of AUC, that is, the area under the ROC curve. Therefore, in scenarios where privacy protection of label data is required, according to various embodiments of the present disclosure, it is possible to allow service nodes to determine more accurate performance indicators while obtaining differential privacy protection of data.

[0141] In some embodiments, in addition to AUC, other performance indicators may also be calculated. The service node 120 may also correct the error values ​​of these performance indicators by setting other mapping relationships to obtain more accurate performance indicator values.

[0142] Figure 4 A flow chart showing a process 400 for model performance evaluation at a client node according to some embodiments of the present disclosure. The process 400 may be implemented at the client node 110.

[0143] In block 410, the client node 110 obtains a plurality of prediction scores output by the machine learning model for a plurality of data samples, wherein the plurality of prediction scores respectively indicate the prediction probability that the plurality of data samples belong to the first category or the second category.

[0144] In block 420, the client node 110 modifies the plurality of true value labels based on the random response mechanism to obtain a plurality of protected labels. The plurality of true value labels respectively mark the plurality of data samples as belonging to the first category or the second category.

[0145] At block 430, the client node 110 determines error metric information associated with a predetermined performance indicator of the machine learning model based on the plurality of protected tags and the plurality of prediction scores. At block 440, the client node 110 sends the error metric information to the service node.

[0146] In some embodiments, determining the error metric information includes: determining a plurality of prediction scores and a plurality of protected labels as the error metric information.

[0147] In some embodiments, the plurality of prediction scores are determined as a first portion of the error metric information and sent to the service node. In some embodiments, determining the error metric information further comprises: after sending the plurality of prediction scores to the service node, receiving from the service node a ranking result of each of the plurality of prediction scores in a prediction score set, the prediction score set comprising prediction scores sent by a plurality of client nodes, the plurality of client nodes comprising the client node; and determining a second portion of the error metric information based on the plurality of protected tags and the ranking result of each of the plurality of prediction scores.

[0148] In some embodiments, determining the second part of the error metric information includes: determining a first number of a first class of protected tags among a plurality of protected tags, the first class of protected tags indicating that the corresponding data samples belong to a first category; determining a second number of a second class of protected tags among a plurality of protected tags, the second class of protected tags indicating that the corresponding data samples belong to a second category; and determining a third number of prediction scores in the prediction score set that the prediction score of the data sample corresponding to the first class of protected tags exceeds based on the ranking results of each of the plurality of prediction scores.

[0149] In some embodiments, sending the error metric information includes: adjusting an order of the plurality of prediction scores; and sending the plurality of prediction scores to the service node in the adjusted order.

[0150] In some embodiments, the predetermined performance metric includes at least an area under a receiver operating characteristic (ROC) curve (AUC).

[0151] Figure 5 A flow chart illustrating a process 500 for model performance evaluation at a service node according to some embodiments of the present disclosure is shown. The process 500 may be implemented at the service node 120 .

[0152] At block 510, the service node 120 receives error metric information related to a predetermined performance indicator of a machine learning model from a plurality of client nodes, respectively. The error metric information is determined by the respective client nodes based on a plurality of protected tags, respectively. The plurality of protected tags are generated by applying a random response mechanism to a plurality of true value tags.

[0153] At block 520, the service node 120 determines an error value of a predetermined performance indicator based on the error metric information. At block 530, the service node 120 determines a corrected value of the predetermined performance indicator by correcting the error value.

[0154] In some embodiments, receiving error metric information includes: for a given client node among a plurality of client nodes, receiving a plurality of protected labels and a plurality of prediction scores from the given client node, the plurality of prediction scores being determined by a machine learning model based on a plurality of data samples, and the plurality of prediction scores respectively indicating a predicted probability that the plurality of data samples belong to a first category or a second category.

[0155] In some embodiments, determining an error value of a predetermined performance indicator includes: determining a first total number of first-category protected tags and a second total number of second-category protected tags in a set of protected tags received from multiple client nodes, the first-category protected tags indicating that the corresponding data samples belong to the first category, and the second-category protected tags indicating that the corresponding data samples belong to the second category; sorting a set of prediction scores received from multiple client nodes; determining a third total number of prediction scores in the set of prediction scores that the prediction scores of the data samples corresponding to the first-category protected tags exceed based on the sorting results of the respective prediction scores in the set of prediction scores; and calculating the error value of the predetermined performance indicator based on the first total number, the second total number, and the third total number.

[0156] In some embodiments, receiving error metric information includes: for a given client node among a plurality of client nodes, receiving a plurality of prediction scores from the given client node as a first part in the error metric information, the plurality of prediction scores being determined by a machine learning model based on a plurality of data samples, and the plurality of prediction scores respectively indicating a predicted probability that the plurality of data samples belong to a first category or a second category.

[0157] In some embodiments, process 500 also includes: determining a ranking result of multiple prediction scores from a given client node in a prediction score set, the prediction score set including prediction scores sent by multiple client nodes; and sending the ranking result of the multiple prediction scores to the given client node.

[0158] In some embodiments, receiving error metric information also includes: receiving from a given client node a first number of a first class of protected tags among a plurality of protected tags at a given client node, and a second number of a second class of protected tags among a plurality of protected tags, the first class of protected tags indicating that the corresponding data samples belong to a first category, and the second class of protected tags indicating that the corresponding data samples belong to a second category; and receiving from the given client node a third number, the third number indicating the number of prediction scores that the prediction scores of the data samples corresponding to the first class of protected tags exceed in the prediction score set.

[0159] In some embodiments, determining an error value of a predetermined performance indicator includes: obtaining a first total number of first-category protected tags by aggregating a first number of first-category protected tags received from multiple client nodes; obtaining a second total number of second-category protected tags by aggregating a second number of second-category protected tags received from multiple client nodes; obtaining a third total number of predicted scores that the predicted scores of data samples corresponding to the first-category protected tags exceed in the predicted score set by aggregating a third number of predicted scores received from multiple client nodes; and calculating the error value of the predetermined performance indicator based on the first total number, the second total number, and the third total number.

[0160] In some embodiments, determining a correction value of a predetermined performance indicator includes: obtaining a first total number of first-category protected tags and a second total number of second-category protected tags in a protected tag set of multiple client nodes, the first-category protected tags indicating that corresponding data samples belong to a first category, and the second-category protected tags indicating that corresponding data samples belong to a second category; determining a mapping relationship between an error value and a correction value of the predetermined performance indicator based on the first total number and the second total number; and calculating the correction value of the predetermined performance indicator from the error value based on the mapping relationship.

[0161] Figure 6 A block diagram of an apparatus 600 for model performance evaluation at a client node according to some embodiments of the present disclosure is shown. The apparatus 600 may be implemented as or included in the client node 110. Each module / component in the apparatus 600 may be implemented by hardware, software, firmware, or any combination thereof.

[0162] As shown in the figure, the device 600 includes a score acquisition module 610, which is configured to obtain multiple prediction scores output by the machine learning model for multiple data samples. The multiple prediction scores respectively indicate the predicted probabilities that the multiple data samples belong to the first category or the second category. The device 600 also includes a label modification module 620, which is configured to modify multiple true value labels based on a random response mechanism to obtain multiple protected labels, and the multiple true value labels respectively mark multiple data samples as belonging to the first category or belonging to the second category. In addition, the device 600 also includes an information determination module 630, which is configured to determine error metric information related to a predetermined performance indicator of the machine learning model based on multiple protected labels and multiple prediction scores; and an information sending module 640, which is configured to send error metric information to a service node.

[0163] In some embodiments, the information determination module 630 includes: a first determination module configured to determine a plurality of prediction scores and a plurality of protected labels as error metric information.

[0164] In some embodiments, the plurality of prediction scores are determined as a first portion of the error metric information and sent to the service node. In some embodiments, the information determination module 630 includes: a ranking result receiving module configured to receive, from the service node, a ranking result of each of the plurality of prediction scores in a prediction score set after sending the plurality of prediction scores to the service node, the prediction score set including prediction scores sent by the plurality of client nodes, the plurality of client nodes including the client node; and a second determination module configured to determine a second portion of the error metric information based on the ranking results of each of the plurality of protected tags and the plurality of prediction scores.

[0165] In some embodiments, the second determination module includes: a first number determination module, configured to determine a first number of a first class of protected tags among a plurality of protected tags, the first class of protected tags indicating that the corresponding data samples belong to a first category; a second number determination module, configured to determine a second number of a second class of protected tags among a plurality of protected tags, the second class of protected tags indicating that the corresponding data samples belong to a second category; and a third number determination module, configured to determine, based on the ranking results of each of the plurality of prediction scores, a third number of prediction scores that the prediction score of the data sample corresponding to the first class of protected tags exceeds in the prediction score set.

[0166] In some embodiments, the information sending module 640 includes: an order adjustment module configured to adjust the order of multiple prediction scores; and an order sending module configured to send the multiple prediction scores to the service node in the adjusted order.

[0167] In some embodiments, the predetermined performance metric includes at least an area under a receiver operating characteristic (ROC) curve (AUC).

[0168] Figure 7 A block diagram of an apparatus 700 for model performance evaluation at a service node according to some embodiments of the present disclosure is shown. The apparatus 700 may be implemented as or included in the service node 120. Each module / component in the apparatus 700 may be implemented by hardware, software, firmware, or any combination thereof.

[0169] As shown in the figure, the device 700 includes an information receiving module 710, which is configured to receive error measurement information related to predetermined performance indicators of the machine learning model from multiple client nodes. The error measurement information is determined by the corresponding client nodes based on their respective multiple protected tags. The multiple protected tags are generated by applying a random response mechanism to multiple true value tags. The device 700 also includes an indicator determination module 720, which is configured to determine the error value of the predetermined performance indicator based on the error measurement information; and an indicator correction module 730, which is configured to determine the correction value of the predetermined performance indicator by correcting the error value.

[0170] In some embodiments, the information receiving module 710 includes: a first receiving module, configured to receive multiple protected labels and multiple prediction scores from a given client node among multiple client nodes, the multiple prediction scores are determined by a machine learning model based on multiple data samples, and the multiple prediction scores respectively indicate the predicted probability that the multiple data samples belong to the first category or the second category.

[0171] In some embodiments, the indicator determination module 720 includes: a first total number determination module, configured to determine a first total number of first-category protected tags and a second total number of second-category protected tags in a protected tag set received from multiple client nodes, the first-category protected tags indicating that the corresponding data samples belong to the first category, and the second-category protected tags indicating that the corresponding data samples belong to the second category; a sorting module, configured to sort the prediction score set received from multiple client nodes; a second total number determination module, configured to determine, based on the sorting results of the respective prediction scores in the prediction score set, a third total number of prediction scores that the prediction scores of the data samples corresponding to the first-category protected tags exceed in the prediction score set; and a first indicator determination module based on the total number, configured to calculate the error value of a predetermined performance indicator based on the first total number, the second total number and the third total number.

[0172] In some embodiments, the information receiving module 710 includes: a second receiving module configured to receive, for a given client node among the plurality of client nodes, a plurality of prediction scores from the given client node as a first part of the error metric information. The plurality of prediction scores are determined by a machine learning model based on a plurality of data samples, and the plurality of prediction scores respectively indicate a prediction probability that the plurality of data samples belong to the first category or the second category.

[0173] In some embodiments, the device 700 also includes: a ranking determination module, configured to determine a ranking result of multiple prediction scores from a given client node in a prediction score set, the prediction score set including prediction scores sent by multiple client nodes; and a second sending module, configured to send the ranking result of the multiple prediction scores to the given client node.

[0174] In some embodiments, the information receiving module 710 also includes: a third receiving module, configured to receive from a given client node a first number of a first class of protected tags among a plurality of protected tags at a given client node, and a second number of a second class of protected tags among a plurality of protected tags, the first class of protected tags indicating that the corresponding data sample belongs to the first category, and the second class of protected tags indicating that the corresponding data sample belongs to the second category; and a fourth receiving module, configured to receive from a given client node a third number, the third number indicating the number of predicted scores that the predicted scores of the data samples corresponding to the first class of protected tags exceed in the predicted score set.

[0175] In some embodiments, the indicator determination module 720 includes: a first aggregation module, configured to obtain a first total number of first-class protected tags by aggregating a first number of first-class protected tags received from multiple client nodes; a second aggregation module, configured to obtain a second total number of second-class protected tags by aggregating a second number of second-class protected tags received from multiple client nodes; a third aggregation module, configured to obtain a third total number of predicted scores that the predicted scores of data samples corresponding to the first-class protected tags exceed in the predicted score set by aggregating a third number of predicted scores received from multiple client nodes; and a second indicator determination module based on the total number, configured to calculate the error value of a predetermined performance indicator based on the first total number, the second total number and the third total number.

[0176] In some embodiments, the indicator correction module 730 includes: a number acquisition module, configured to obtain a first total number of a first category of protected tags and a second total number of a second category of protected tags in a protected tag set of multiple client nodes, the first category of protected tags indicating that the corresponding data sample belongs to the first category, and the second category of protected tags indicating that the corresponding data sample belongs to the second category; a mapping determination module, configured to determine a mapping relationship between an error value and a correction value of a predetermined performance indicator based on the first total number and the second total number; and a correction value determination module, configured to calculate a correction value of a predetermined performance indicator from the error value based on the mapping relationship.

[0177] Figure 8 800 is a block diagram of a computing device / system 800 in which one or more embodiments of the present disclosure may be implemented. Figure 8 The computing device / system 800 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 8 The computing device / system 800 shown may be used to implement Figure 1 The client node 110 or the service node 120 of FIG.

[0178] like Figure 8As shown, computing device / system 800 is in the form of a general computing device. Components of computing device / system 800 may include, but are not limited to, one or more processors or processing units 810, memory 820, storage device 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. Processing unit 810 may be an actual or virtual processor and is capable of performing various processes according to a program stored in memory 820. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to increase the parallel processing capabilities of computing device / system 800.

[0179] The computing device / system 800 typically includes a plurality of computer storage media. Such media may be any available media accessible to the computing device / system 800, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 820 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 830 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the computing device / system 800.

[0180] The computing device / system 800 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 8 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 820 may include a computer program product 825 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0181] The communication unit 840 enables communication with other computing devices via a communication medium. Additionally, the functions of the components of the computing device / system 800 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the computing device / system 800 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0182] Input device 850 may be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 860 may be one or more output devices, such as a display, speaker, printer, etc. Computing device / system 800 may also communicate with one or more external devices (not shown) as needed, such as storage devices, display devices, etc., with one or more devices that allow a user to interact with computing device / system 800, or with any device that allows computing device / system 800 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.) through communication unit 840. Such communication may be performed via an input / output (I / O) interface (not shown).

[0183] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions or a computer program are stored, wherein the computer-executable instructions or the computer program are executed by a processor to implement the method described above.

[0184] According to an exemplary implementation of the present disclosure, a computer program product is also provided. The computer program product is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0185] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0186] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0187] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0188] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0189] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for evaluating model performance, comprising: At the client node, a plurality of prediction scores output by the machine learning model for a plurality of data samples are obtained, wherein the plurality of prediction scores respectively indicate a prediction probability that the plurality of data samples belong to a first category or a second category; Modifying a plurality of true value labels based on a random response mechanism to obtain a plurality of protected labels, wherein the plurality of true value labels respectively mark the plurality of data samples as belonging to the first category or the second category; determining error metric information associated with a predetermined performance indicator of the machine learning model based on the plurality of protected labels and the plurality of prediction scores; as well as The error metric information is sent to a service node.

2. The method of claim 1 , wherein determining the error metric information comprises: The plurality of prediction scores and the plurality of protected labels are determined as the error metric information.

3. The method of claim 1 , wherein the plurality of prediction scores are determined as a first portion of the error metric information and sent to the service node, and wherein determining the error metric information further comprises: After sending the plurality of prediction scores to the service node, receiving from the service node a ranking result of each of the plurality of prediction scores in a set of prediction scores, the set of prediction scores including prediction scores sent by a plurality of client nodes including the client node; as well as Based on the ranking results of each of the plurality of protected tags and the plurality of prediction scores, a second portion of the error metric information is determined.

4. The method of claim 3, wherein determining the second portion of the error metric information comprises: determining a first number of first-category protected tags among the plurality of protected tags, the first-category protected tags indicating that corresponding data samples belong to the first category; determining a second number of second-class protected tags among the plurality of protected tags, the second-class protected tags indicating that the corresponding data sample belongs to the second class; as well as Based on the ranking results of each of the plurality of prediction scores, a third number of prediction scores in the prediction score set that the prediction score of the data sample corresponding to the first class of protected labels exceeds is determined.

5. The method of claim 3, wherein sending the error metric information comprises: adjusting the order of the plurality of prediction scores; as well as The multiple prediction scores are sent to the service node in the adjusted order.

6. The method according to any one of claims 1 to 5, wherein the predetermined performance indicator comprises at least the area under the curve (AUC) of a receiver operating characteristic (ROC) curve.

7. A method for evaluating model performance, comprising: At the service node, error metric information related to a predetermined performance indicator of the machine learning model is received from a plurality of client nodes, respectively, the error metric information being determined by the corresponding client nodes based on a respective plurality of protected tags, the plurality of protected tags being generated by applying a random response mechanism to a plurality of true value tags; Determining an error value of the predetermined performance indicator based on the error metric information; as well as A corrected value of the predetermined performance indicator is determined by correcting the error value.

8. The method of claim 7, wherein receiving the error metric information comprises: For a given client node among the plurality of client nodes, The multiple protected tags and multiple prediction scores are received from the given client node, where the multiple prediction scores are determined by the machine learning model based on multiple data samples, and the multiple prediction scores respectively indicate the predicted probability that the multiple data samples belong to the first category or the second category.

9. The method of claim 8, wherein determining the error value of the predetermined performance indicator comprises: Determining, in the protected tag sets received from the plurality of client nodes, a first total number of protected tags of a first category indicating that corresponding data samples belong to the first category and a second total number of protected tags of a second category indicating that corresponding data samples belong to the second category; sorting the sets of prediction scores received from the plurality of client nodes; Based on the ranking results of the prediction scores in the prediction score set, determining a third total number of prediction scores in the prediction score set that the prediction scores of the data samples corresponding to the first class of protected labels exceed; as well as The error value of the predetermined performance indicator is calculated based on the first total number, the second total number and the third total number.

10. The method of claim 7, wherein receiving the error metric information comprises: For a given client node among the plurality of client nodes, A plurality of prediction scores are received from the given client node as a first part of the error metric information, the plurality of prediction scores being determined by the machine learning model based on a plurality of data samples, and the plurality of prediction scores respectively indicating a predicted probability that the plurality of data samples belong to a first category or a second category.

11. The method according to claim 10, further comprising: determining a ranking result of the plurality of prediction scores from the given client node in a set of prediction scores, the set of prediction scores comprising prediction scores sent by the plurality of client nodes; as well as The ranking result of the plurality of prediction scores is sent to the given client node.

12. The method of claim 11, wherein receiving the error metric information further comprises: receiving, from the given client node, a first number of protected tags of a first class among the plurality of protected tags at the given client node, and a second number of protected tags of a second class among the plurality of protected tags, the first class of protected tags indicating that the corresponding data sample belongs to the first category, the second class of protected tags indicating that the corresponding data sample belongs to the second category; as well as A third number is received from the given client node, the third number indicating the number of prediction scores in the set of prediction scores that the prediction score for the data samples corresponding to the first class of protected labels exceeds.

13. The method of claim 12, wherein determining the error value of the predetermined performance indicator comprises: Obtaining a first total number of the first class of protected tags by aggregating the first number of the first class of protected tags received from the plurality of client nodes; Obtaining a second total number of the second type of protected tags by aggregating the second numbers of the second type of protected tags received from the plurality of client nodes; Obtaining, by aggregating the third number of prediction scores received from the plurality of client nodes, a third total number of prediction scores that the prediction scores of the data samples corresponding to the first class of protected tags exceed in the set of prediction scores; as well as The error value of the predetermined performance indicator is calculated based on the first total number, the second total number and the third total number.

14. The method of claim 7, wherein determining the correction value of the predetermined performance indicator comprises: Obtaining a first total number of first-category protected tags and a second total number of second-category protected tags in a protected tag set of the plurality of client nodes, wherein the first-category protected tags indicate that corresponding data samples belong to a first category, and the second-category protected tags indicate that corresponding data samples belong to a second category; Determine a mapping relationship between an error value and a correction value of the predetermined performance indicator based on the first total number and the second total number; as well as Based on the mapping relationship, the correction value of the predetermined performance indicator is calculated from the error value.

15. A device for model performance evaluation, comprising: A score obtaining module is configured to obtain a plurality of prediction scores output by the machine learning model for a plurality of data samples, wherein the plurality of prediction scores respectively indicate a prediction probability that the plurality of data samples belong to the first category or the second category; a label modification module, configured to modify a plurality of true value labels based on a random response mechanism to obtain a plurality of protected labels, wherein the plurality of true value labels respectively mark the plurality of data samples as belonging to the first category or the second category; an information determination module configured to determine error metric information associated with a predetermined performance indicator of the machine learning model based on the plurality of protected tags and the plurality of prediction scores; as well as The information sending module is configured to send the error metric information to the service node.

16. A device for model performance evaluation, comprising: An information receiving module is configured to receive error metric information related to a predetermined performance indicator of the machine learning model from a plurality of client nodes, respectively, the error metric information being determined by the corresponding client nodes based on a respective plurality of protected tags, the plurality of protected tags being generated by applying a random response mechanism to a plurality of true value tags; an indicator determination module, configured to determine an error value of the predetermined performance indicator based on the error metric information; as well as The indicator correction module is configured to determine a correction value of the predetermined performance indicator by correcting the error value.

17. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the device to perform the method according to any one of claims 1 to 6 when executed by the at least one processing unit.

18. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the device to perform a method according to any one of claims 7 to 14 when executed by the at least one processing unit.

19. A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method according to any one of claims 1 to 6.

20. A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method according to any one of claims 7 to 14.

21. A computer program product tangibly stored in a computer storage medium and comprising computer executable instructions which, when executed by a device, cause the device to perform the method of any one of claims 1 to 6 or the method of any one of claims 7 to 14.

Images