Method and device for determining fairness influence of model, electronic equipment and medium

Abstract

The embodiment of the invention relates to a method and device for determining the influence of a sample on model fairness, electronic equipment and a medium. The method includes generating anti-fact samples by adjusting original samples in a set of original samples, wherein the set of original samples is used to generate an original model that performs a classification task. The method further includes determining a fairness indicator of the original model on the verification sample set. In addition, the method further comprises the step of determining the fairness influence of the original samples on the original model based on the fairness indexes, the original samples and the anti-fact samples. According to the method, the influence of each training sample on the fairness of the model can be determined, and a practitioner is helped to accurately position a set of samples with the maximum influence on the fairness of the model, so that the practitioner can improve the fairness of the model by repairing the training samples.

Description

Technical Field

[0001] The present disclosure generally relates to the field of artificial intelligence, and more specifically, to methods, devices, electronic devices, and media for determining the impact of samples on model fairness. Background Art

[0002] As AI applications continue to develop and become more popular, people are beginning to pay attention to the impact of algorithms and models on social justice. Unfair treatment of individuals with different group attributes (also known as sensitive attributes) may lead to exacerbated social inequality. Therefore, there is an urgent need to ensure the fairness of models to protect the rights and interests of individuals and groups. In the field of AI, fairness can involve many aspects. For example, when training models, it is necessary to ensure the representativeness and diversity of training samples, and to ensure that algorithms do not favor or discriminate against individuals from different groups. Summary of the invention

[0003] Embodiments of the present disclosure provide a method, apparatus, electronic device, and medium for determining the impact of a sample on model fairness. For a set of training samples used to train a model, the method provided by an embodiment of the present disclosure can adjust the original training sample from multiple aspects to generate a counterfactual sample corresponding to the sample. Then, the method can calculate the impact on model fairness when the counterfactual sample is used to train the model. In this way, the method provided by an embodiment of the present disclosure can determine the impact of each training sample on model fairness, helping practitioners to accurately locate the set of samples that have the greatest impact on model fairness, so that practitioners can improve the fairness of the model by repairing these training samples.

[0004] In a first aspect of an embodiment of the present disclosure, a method for determining the impact of a sample on model fairness is provided. The method includes generating a counterfactual sample by adjusting an original sample in an original sample set, the original sample set being used to generate an original model for performing a classification task. The method also includes determining a fairness index of the original model on a verification sample set. In addition, the method also includes determining the fairness impact of the original sample on the original model based on the fairness index, the original sample, and the counterfactual sample.

[0005] In a second aspect of an embodiment of the present disclosure, a device for determining the impact of a sample on model fairness is provided. The device includes a counterfactual sample generation module configured to generate a counterfactual sample by adjusting an original sample in an original sample set, the original sample set being used to generate an original model for performing a classification task. The device also includes a fairness index determination module configured to determine a fairness index of the original model on a verification sample set. In addition, the device also includes a fairness impact determination module configured to determine the fairness impact of the original sample on the original model based on the fairness index, the original sample, and the counterfactual sample.

[0006] In a third aspect of an embodiment of the present disclosure, an electronic device is provided. The electronic device includes one or more processors; and a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement a method for determining the impact of samples on model fairness. The method includes generating counterfactual samples by adjusting the original samples in the original sample set, and the original sample set is used to generate an original model for performing a classification task. The method also includes determining a fairness index of the original model on the verification sample set. In addition, the method also includes determining the fairness impact of the original sample on the original model based on the fairness index, the original sample, and the counterfactual sample.

[0007] In a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, a method for determining the impact of a sample on model fairness is implemented. The method includes generating a counterfactual sample by adjusting an original sample in an original sample set, and the original sample set is used to generate an original model for performing a classification task. The method also includes determining a fairness index of the original model on a verification sample set. In addition, the method also includes determining the fairness impact of the original sample on the original model based on the fairness index, the original sample, and the counterfactual sample.

[0008] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1A schematic diagram illustrating an example environment in which various embodiments of the present disclosure may be implemented;

[0011] Figure 2 A flowchart of a method for determining the impact of a sample on model fairness according to some embodiments of the present disclosure is shown;

[0012] Figures 3A to 3D shows an overall causal graph involving concepts according to some embodiments of the present disclosure, and a causal graph corresponding to intervention group attributes, intervention features, and intervention labels;

[0013] Figure 4 A schematic diagram showing a process of improving model fairness by intervening in group attributes according to some embodiments of the present disclosure;

[0014] Figures 5A to 5C A schematic diagram showing a process of generating counterfactual samples by intervening group attributes, intervening features, and intervening labels according to some embodiments of the present disclosure;

[0015] Figure 6 A block diagram showing an apparatus for determining the impact of a sample on model fairness according to some embodiments of the present disclosure; and

[0016] Figure 7 A block diagram of a device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0017] It is understood that all user-related data involved in this technical solution should be obtained and used after the user's authorization. This means that in this technical solution, if the user's personal information needs to be used, the user's explicit consent and authorization are required before obtaining this data, otherwise the relevant data will not be collected and used. It should also be understood that when implementing this technical solution, relevant laws and regulations should be strictly observed during the collection, use and storage of data, and necessary technologies and measures should be taken to protect the user's data security and ensure the safe use of data.

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects, unless explicitly stated. Other explicit and implicit definitions may also be included below.

[0020] A fundamental question about fairness in machine learning is the source of unfairness. In practice, this is also one of the first questions practitioners ask after calculating fairness metrics and discovering model unfairness. Although this question sounds simple, it is difficult to determine the exact source of unfairness in the machine learning process. There can be many possible reasons for unfairness in AI models, including insufficient training data, lack of features, wrong prediction targets, or inaccurate measurements of input features. These problems are not so easy to solve even for the most experienced AI researchers and engineers.

[0021] There are many sources of unfairness, including data sampling bias or underrepresentation, data labeling bias, model architecture (or feature representation), distribution shift, etc. The embodiments of the present disclosure focus on the most important and obvious source of bias - training samples. Because if the training samples of the model are biased, it is difficult for the model trained with these training samples to have high fairness unless a high cost is paid to modify the model. In particular, the embodiments of the present disclosure focus on how the training samples will affect the unfairness of the model by focusing on the following questions: if the training samples of the model are collected from different groups (e.g., demographic groups), how will the fairness indicators of the model change; if the training samples of the model are labeled with different values, how will the fairness indicators of the model change; if the values ​​of certain features of the training samples of the model are changed, how will the fairness indicators of the model change. Answering these questions can help practitioners explain why the model is unfair in terms of training data, repair training data to improve fairness, and detect biased or noisy training labels, underrepresented groups, and features that undermine fairness.

[0022] To this end, according to an embodiment of the present disclosure, a scheme for determining the impact of a sample on model fairness is provided. The scheme involves an original sample set, a verification sample set, and an original model trained using the original sample set. The scheme generates counterfactual samples by adjusting the original samples in the original sample set, and then first determines the fairness index of the original model on the verification sample set, and then determines the fairness impact of the original sample on the original model based on the fairness index, the original sample, and the counterfactual sample. In this way, the impact of each training sample on the fairness of the model can be determined, helping practitioners to accurately locate those training samples that have the greatest impact on the fairness of the model, so that practitioners can improve the fairness of the model by repairing these training samples.

[0023] Figure 1 1 is a schematic diagram of an example environment 100 in which various embodiments of the present disclosure can be implemented. Figure 1 As shown, the environment 100 includes an original sample set 102, and the original sample set 102 includes original samples 104, 106, and 108, etc. The original sample set 102 may include any number of original samples. Figure 1 Only three original samples are shown, but the number of original samples in the original sample set 102 is not intended to be limited. The environment 100 also includes an original model 110, which is a model based on the original sample set 102 and trained using a loss function 112. The original model 110 is a classification model for performing classification tasks, and the classification tasks may include, for example, image classification, video classification, content classification, etc. For example, the machine learning task performed by the original model 110 may be to predict the user's interest preferences (for example, predict whether the user is interested in content related to cars). Each original sample in the original sample set 102 has group attributes, features, and labels. For example, Figure 1 As shown, the original sample 104 includes group attributes 114, features 116 and labels 118. The group attribute 114 may be, for example, the user's residential area, etc., the feature 116 may be, for example, whether the number of outings in the past year exceeds 10 times, etc., and the label 118 may be, for example, whether the user is interested in or not interested in cars, etc. In the embodiments of the present disclosure, the original sample set may include various structured data and unstructured data (for example, image data, text data, voice data), etc., and the embodiments of the present disclosure are not limited to this, as long as the data is suitable for training, prediction, etc. of the artificial intelligence model.

[0024] In this article, group attributes, features, and labels are collectively referred to as "concepts", and concepts refer to categorical variables that describe data characteristics. For example, in some embodiments, group attribute 114 can be selected as a concept, and then counterfactual interventions are performed on group attribute 114 to answer what impact the fairness of the model will have if training data is collected from another different group. In some embodiments, feature 116 can also be selected as a concept, and then counterfactual interventions are performed on feature 116 to answer how the fairness index of the model will change if the values ​​of certain features of the training samples of the model are changed. In some embodiments, label 118 can also be selected as a concept, and then counterfactual interventions are performed on label 118 to answer what impact the fairness of the model will have if the value of the label of the training sample is changed. In some embodiments, the existence of training samples can also be selected as a concept, and then the impact on the fairness of the model can be observed by, for example, marking the original sample 104 as removed. For example, in environment 100, counterfactual intervention can be performed on the population attribute 114, feature 116 or label 118 of the original sample 104 to generate a counterfactual sample 124, and then an adjusted sample set 122 is generated by replacing the original sample 104 with the counterfactual sample 124, and the adjusted sample set 122 includes the counterfactual sample 124, the original sample 106 and the original sample 108.

[0025] like Figure 1 As shown, the counterfactual model 130 is a model corresponding to the adjustment sample set 122. The counterfactual model 130 is not necessarily a model that has been truly trained, and it may also represent that if the adjustment sample set 122 is used for training, the counterfactual model 130 will be obtained. In the environment 100, the fairness index 132 of the original model 110 on the verification sample set 126 and the fairness index 134 of the counterfactual model 130 on the verification sample set 126 may be determined respectively, and then the fairness impact value 136 is determined based on the fairness indexes 132 and 134.

[0026] Figure 2 FIG. 2 is a flowchart of a method 200 for determining the impact of a sample on model fairness according to some embodiments of the present disclosure. Figure 2 As shown, at block 202, method 200 generates counterfactual samples by adjusting original samples in an original sample set, the original sample set being used to generate an original model for performing a classification task. Figure 1 In the illustrated environment 100 , an original sample set 102 is used to generate an original model 110 , and the method 200 may intervene in the concepts of the original samples 104 in the original sample set 102 (e.g., group attributes 114 , features 116 , labels 118 , or marking the original samples 104 as removed) to generate counterfactual samples 124 .

[0027] At block 204, method 200 may determine a fairness metric for the original model on the validation sample set. Figure 1 In the environment 100 shown, the method 200 can determine the fairness index 132 of the original model 110 on the validation sample set 126. The fairness index 132 is a value that can quantify the fairness of the original model 110 on the validation sample set 126, and an approximate solution of the fairness index 132 can be obtained by calculating the replacement loss of the original model 110 on the validation sample set 126. The validation sample set 126 is a sample set independent of the original sample set 102, and the validation sample set 126 includes representative samples from various groups. Calculating the fairness index 132 on the validation sample set 126 can provide an evaluation of the fairness performance of the original model 110 on real data, thereby checking the performance of the model in a real environment.

[0028] At block 206, method 200 may determine the fairness impact of the original sample on the original model based on the fairness index, the original sample, and the counterfactual sample. Figure 1 In the illustrated environment 100, the fairness impact of the intervention on the original sample 104 on the original model 110 can be represented by a fairness impact value 136, which indicates the difference between the fairness index 132 of the original model 110 on the validation sample set 126 and the fairness index 134 of the counterfactual model 130 on the validation sample set 126. However, in some embodiments, the counterfactual model 130 is not actually obtained through training, so the fairness index 134 cannot be directly determined. Therefore, the method 200 can calculate an approximate solution of the fairness impact value 136 based on the fairness index 132 of the original model 110 on the validation sample set 126, the original sample 104, and the counterfactual sample 124.

[0029] In this way, method 200 can determine the impact of each original sample on the fairness of the original model 110 by adjusting or intervening in each original sample in the original sample set 102, thereby helping practitioners to accurately locate the set of samples in the original sample set 102 that have the greatest impact on the fairness of the original model 110, so that practitioners can improve the fairness of the model by repairing these training samples.

[0030] The following will be passed Figures 3A to 3D The overall causal graph of the concepts involved in some embodiments of the present disclosure and multiple causal graphs corresponding to intervention group attributes, intervention features and intervention labels are introduced. Then, the detailed process of determining the fairness impact value is introduced. In addition, the following examples are given by way of example. Figure 4The process of improving model fairness by intervening in group attributes according to some embodiments of the present disclosure is introduced. Figures 5A to 5C Different processes for generating counterfactual samples according to some embodiments of the present disclosure are introduced.

[0031] Figure 3A The overall causal graph 300 involving concepts according to some embodiments of the present disclosure is shown. In the causal graph, nodes represent variables or factors, and variables can be observable variables or unobservable potential variables. Edges represent causal relationships or dependencies between variables. Figure 3A As shown, training sample 302 includes group attribute 304, feature 306, and label 308, wherein feature 306 may include multiple features. In other places in this article, A is also used to represent group attribute, X is used to represent feature, and Y is used to represent label. Concept 310 is a sample-level classification attribute associated with training sample 302. represents a concept, where C is a discrete concept encoded in the data (X, Y, A). The causal graph 300 shows how to quantify the effect of using counterfactual samples (e.g., Figure 1 The counterfactual samples 124 in the example are used to replace the original samples (e.g., Figure 1 The original sample 104 in the Figure 3A As shown, in the causal graph 300 , feature 306 depends on group attribute 304 , label 308 depends on feature 306 and group attribute 304 , model 312 depends on feature 306 and label 308 , and fairness 316 depends on model 312 and validation sample set 314 .

[0032] Figure 3B A causal diagram 320 is shown when intervening the group attributes of the sample according to some embodiments of the present disclosure. Group attributes are closely related to fairness indicators because group attributes can control the sampling distribution of each group. Figure 3BAs shown, in the causal graph 320, the group attribute 324 is selected as the concept to be intervened, which means exploring what impact will be produced on fairness if the training samples 322 come from different groups. When the group attribute 302 is intervened (for example, changed from one value to another value), the intervention can affect the value of each feature 326 (as shown by arrow 331), and the intervention can also affect the value of the label 328 (as shown by arrow 333). For example, when the group attribute of the user is changed, it can affect whether the number of outings of the user in the past year exceeds 10 times (i.e., the feature), and it can also affect whether the user is interested in car-related content (i.e., the label). Since the model 332 depends on the feature 326 and the label 328, when the feature 326 and the label 328 change, the model 332 also changes accordingly, and the fairness 336 also changes accordingly.

[0033] Figure 3C A cause-effect diagram 340 is shown when intervening on features of a sample according to some embodiments of the present disclosure. Figure 3C As shown, in the causal graph 340, the features of the training sample 342 are selected as the concept 350. When a certain feature is intervened, the group attribute 344 of the training sample 342 will not be affected, because the group attribute 344 does not depend on the feature 346. However, the intervention of one feature of the training sample 342 can affect the values ​​of all other features 346 (as shown by arrow 351). In addition, since the feature 346 is affected, the value of the label 348 is also affected at the same time (as shown by arrow 353). For example, when the number of outings of the user exceeds 10 times in a year (i.e., the feature) is changed from "yes" to "no", other features of the user and whether the user is interested in car-related content (i.e., the label) can be affected. Since the model 352 depends on the feature 346 and the label 348, when the feature 346 and the label 348 change, the model 352 also changes accordingly, and the fairness 356 also changes accordingly. The intervention of the feature means that when the collected training sample set has different values ​​for the same feature, the model will become more fair.

[0034] Figure 3D A cause-effect diagram 340 is shown when intervening on features of a sample according to some embodiments of the present disclosure. Figure 3CAs shown, in the causal diagram 360, the label 368 of the training sample 362 is selected as the concept. Since neither the group attribute 364 nor the feature 366 depends on the label 368, when the label 368 is intervened, neither the group attribute 364 nor the feature 366 of the training sample 362 is affected. However, since the model 362 depends on the label 368, the intervention will affect the model 362 (as shown by the arrow 362), and the fairness 366 will change accordingly. In many cases, there is uncertainty in the label of the sample, and sometimes the observed label may be encoded with noise, inaccurately marked, or have subjective bias, so the intervention on the label means the counterfactual effect, that is, the impact of the intervention on the result is evaluated by comparing the observed reality and the assumed alternative situation.

[0035] Apart from Figures 3B to 3D In addition to the three cases shown, the concept can also be to remove the training sample from the data set. In some embodiments, it can be considered that each sample has a selection variable whose value is 0 or 1. The value of the selection variable of the sample that appears in the training sample set is 1. If the value is changed to 0, it means that the sample is counterfactually excluded from the training sample set. In this way, the impact of the intervention on the fairness of the model can be evaluated by comparing the presence or absence of a certain sample in the training sample set.

[0036] In some embodiments, in order to determine the impact of the intervened original sample on the fairness of the model, the loss for the original sample (also called the first loss) can be determined based on the original sample and using the loss function used to generate the original model, the loss for the counterfactual sample (also called the second loss) can be determined based on the counterfactual sample and using the loss function, and based on the fairness indicator of the original model on the validation data set, the loss for the original sample, and the loss for the counterfactual sample, a fairness impact value indicating the impact of the original sample on the fairness of the original model is determined.

[0037] In some embodiments, in order to determine the fairness impact value, the loss function used to train the original model can be used to generate multiple losses corresponding to multiple original samples in the original sample set, and the Hessian Matrix for the original model is determined based on the multiple losses. And based on the Hessian matrix, the fairness indicator of the original model on the verification data set, the loss for the original samples, and the loss for the counterfactual samples, the fairness impact value is determined.

[0038] In this article, we use To represent the training sample set, represents the i-th training sample, express Features, express , and use To represent the validation sample set, represents the i-th validation sample, express Features, express Assume that the model is parameterized as θ∈Θ and there are samples indexed as A subset of the training sample set. If for each sample Assign weight w i ∈[0, 1] to perturb the group Use represents the generated counterfactual model. The reweighted groups in the training data Impact on model fairness Limited to the original model (trained on the full training data) Counterfactual Model The difference in fairness indicators between It can be expressed by equation (1):

[0039]

[0040] Among them l fair Represents the fairness indicator.

[0041] According to equation (1), a closed-form solution for the fairness effect can be derived. The first-order approximation of can be expressed as follows:

[0042]

[0043] in For the original model The Hessian matrix of And l is used to train the original model The original loss function of (for example, it can be the cross entropy loss function in classification tasks).

[0044] Fairness Indicators The trained model can be quantized In some embodiments, the model can be used to obtain the fairness of the alternative loss on the validation sample set. In these embodiments, the classifier corresponding to the model θ can be represented as h θ , then the group fairness indicator - Demographic Parity (DP) can be expressed as It is approximately in the form of the following equation (4) (assuming that both the group attribute A and the classification task are binary):

[0045]

[0046] where g is the logit of the predicted probability for class 1, and a i is the population attribute of the i-th sample.

[0047] To quantify the counterfactual effect when changing the concept c for each sample (x, y, a), we can use To represent the counterfactual samples generated by intervening concept c. Given training samples It can be defined by the following equation (5) when the concept C = c' is intervened, and the sample The corresponding counterfactual sample:

[0048]

[0049] where do(·) represents the do operation in the causal model.

[0050] In the above equation (5), when the concept C overlaps with any of (X, Y, A), the do(·) operation has a higher priority and is assumed to automatically cover other dependencies. For example, when the concept is a group attribute (i.e., C = A), it is expressed by the following equation (6): Has higher priority:

[0051]

[0052] The counterfactual example can be expressed as Then use counterfactual samples in the training dataset To replace the original sample Then, the counterfactual model trained on the adjusted sample set It can be expressed by the following equation (7):

[0053]

[0054] Where R(θ) represents the risk of model θ and ∈ represents a small positive number.

[0055] In the validation sample set D val On, use Represents the original model The fairness index,

[0056] And use Representing counterfactual models The fairness index of the original sample i is: when the concept C of the original sample i is intervened to c′, the fairness impact value of the original sample on the original model is It can be expressed by the following equation (8):

[0057]

[0058] Combined with equation (2), the fairness impact value can be calculated by the following equation (9): The approximate solution of :

[0059]

[0060] Where l is the original model The loss function is The original model The Hessian matrix over the original set of samples.

[0061] In some embodiments, the second term in equation (9) may be calculated using the Hessian vector product (HVP) and the third The product of . Then H can be recursively calculated by the following equation (10): -1 v:

[0062]

[0063] in is the approximated Hessian matrix on random batches, and r is the number of recursive iterations.

[0064] Let t be the last recursive iteration, then the final fairness impact value is in is an alternative loss to the fairness indicator (e.g., Eq. (4)).

[0065] Alternatively, in some embodiments, when intervening multiple counterfactual samples at once, E represents the kth sample among E samples and can be obtained by to calculate the average counterfactual effect.

[0066] In this way, the samples in the original sample set can be intervened or adjusted at the sample level, and there is no need to train the counterfactual model on the adjusted sample set. Instead, the approximate solution of the fairness impact value is determined by calculating the fairness index of the original model on the verification data set, the Hessian matrix for the original model, the loss of the original sample calculated using the original loss function, and the loss of the counterfactual sample calculated using the original loss function. Therefore, the impact of each sample in the original sample set on the fairness of the original model can be determined at the sample level without consuming huge computational costs and resource costs, and the fairness of the model can be improved by repairing samples with large fairness impact values.

[0067] To further illustrate the principle that intervening in sample concepts (e.g., group attributes, features, and labels) can improve model fairness, Figure 4 A schematic diagram of a process 400 for improving model fairness by intervening in group attributes according to some embodiments of the present disclosure is shown by way of example. Figure 4 As shown, the horizontal axis of the bar graph is the characteristics of different groups, and the vertical axis is the frequency of occurrence of different groups in the training sample set, wherein the majority group 402 has the highest frequency of occurrence in the training sample set, and the minority group 404 has the lowest frequency of occurrence in the training sample set. Process 400 can intervene in a portion of the majority group 406, and specifically, can adjust the group attribute of each sample in the group 406 to the group attribute of the minority group 404. By doing so, process 400 is similar to resampling the training sample set, and compared with the sample set before resampling, the frequency of occurrence of samples of different groups in the adjusted sample set is more balanced, thereby improving the fairness of the model.

[0068] In the original sample set (for example, Figure 1 When intervening in the group attributes, features or labels of the original sample 104 in the example, it is necessary to generate new counterfactual samples (for example, Figure 1 In the counterfactual samples 124), when generating counterfactual samples, it is necessary to consider the dependencies between group attributes, features and labels (such as Figures 3A to 3D ). Figures 5A to 5C Different processes for generating counterfactual samples according to some embodiments of the present disclosure are described in detail.

[0069] In some embodiments, when the intervention on the original sample is to change its group attribute, a counterfactual sample can be generated based on the changed group attribute and the original feature set of the original sample (also referred to as the first feature set). In some embodiments, a new feature set (also referred to as the second feature set) of the counterfactual sample can be generated based on the changed group attribute and the original feature set of the original sample using a generative adversarial network (also referred to as a first generative adversarial network), and a label of the counterfactual sample can be generated based on the new feature set of the counterfactual sample using the original model.

[0070] Figure 5A FIG. 5 is a schematic diagram of a process 500 for generating counterfactual samples by intervening in the group attributes of the original samples according to some embodiments of the present disclosure. Figure 5A As shown, the original sample 502 includes group attribute 504, feature 505, feature 506, feature 506 and label 508. It should be understood that the original sample 502 can have any number of group attributes, features and labels. Figure 5A Only one group attribute, three features and one label are shown in FIG, but the number of group attributes, features and labels is not intended to be limited. Figure 5A In the process 500, the population attribute 504 of the original sample 502 is intervened into the population attribute 514. Figure 3B As described in , when the group attribute of the sample is changed, the features and labels of the sample will be affected at the same time. Therefore, process 500 needs to generate new features and labels for the counterfactual sample. Process 500 can input group attribute 514, feature 505, feature 506 and feature 507 into generative adversarial network 509, and generative adversarial network 509 outputs new features 515, feature 516 and feature 517 based on the input information. Then, process 500 can input new features 515, feature 516 and feature 517 into original model 510, and the original model 510 outputs new label 518, thereby obtaining counterfactual sample 512. Group attribute 514 of counterfactual sample 512 is obtained through intervention, while features 515, feature 516, feature 517 and label 518 are new features and new labels generated by generative adversarial network 509 and original model 510.

[0071] exist Figure 5AIn the example, the generative adversarial network 509 is trained for the case where the group attribute of the original sample is changed. In some embodiments, in order to train the generative adversarial network 509, a first original sample subset and a second original sample subset may be determined from the original sample set, the group attribute of the samples in the first original sample subset having a first value, and the group attribute of the samples in the second original sample subset having a second value. Then, a first group sample in the first original sample subset is obtained, and a target sample is generated based on the feature set of the first group sample and the second value of the group attribute. Then, the generative adversarial network may be trained based on the target sample and the second original sample subset.

[0072] In some embodiments, the distribution of feature X can be mapped from group attribute A=a to group attribute A=a′. Then, the original sample set is divided into two groups according to the feature, namely, a first group with feature X|A=a and a second group with feature X|A=a′. Then, the generator G of the generative adversarial network 509 can be trained. a→a′ As the best transfer mapping from feature X|A=a to feature X|A=a′, and at the same time training the discriminator D of the generative adversarial network 509 a→a′ To determine whether a feature is a real sample feature or a feature generated by the generator G a→a′ The generated features are finally trained to make the discriminator D a→a′ Unable to distinguish the mapped samples G a→a′ (X) is still a true sample X|A=a′. The training objective of this process can be expressed by the following equation (11):

[0073]

[0074]

[0075] Where n is the number of training samples and λ is used to balance the generator loss (i.e., ) and the weights of the distance cost function c(·) (i.e., the l2 norm) to ensure that the mapped samples are not far from the original distribution, is the discriminator loss.

[0076] In some embodiments, when the intervention on the original sample is to change its features, a counterfactual sample can be generated based on the changed features and the feature set of the original sample. In some embodiments, another generative adversarial network (also referred to as a second generative adversarial network) can be used to generate a feature set of the counterfactual sample based on the changed features and the feature set of the original sample, and based on the feature set of the counterfactual sample, the original model is used to generate a label of the counterfactual sample.

[0077] Figure 5BFIG. 5 is a schematic diagram showing a process 520 of generating counterfactual samples by intervening in the features of the original samples according to some embodiments of the present disclosure. Figure 5B As shown, the original sample 502 includes group attribute 504, feature 505, feature 506, feature 506, and label 508. Process 520 intervenes feature 505 of the original sample 502 into feature 525 (for example, in the example of predicting whether the user is interested in cars, the value of the feature "whether the number of outings in the past year has exceeded 10 times" can be changed from "yes" to "no"). Figure 3C As described in , when one feature of a sample is changed, other features and labels of the sample will be affected at the same time. Therefore, process 520 needs to generate new features and labels for the counterfactual sample. Process 520 can input features 525, features 506, and features 507 into generative adversarial network 529, and generative adversarial network 529 outputs new features 526 and features 527 based on the information of these inputs. Then, process 520 can input features 525, features 526, and features 527 into original model 510, and the original model 510 outputs a new label 528, thereby obtaining counterfactual sample 522. The group attribute 504 of counterfactual sample 522 remains unchanged, feature 525 is obtained through intervention, and features 526, features 527, and labels 528 are new features and new labels generated by generative adversarial network 509 and original model 510.

[0078] exist Figure 5B In the example, the generative adversarial network 529 is trained for the case where the features of the original samples are changed. In some embodiments, in order to train the generative adversarial network 529, a first subset of original samples and a second subset of original samples may be determined from the original sample set, wherein the designated features of the samples in the first subset of original samples have a first value, and the designated features of the samples in the second subset of original samples have a second value. Then, a first feature sample in the first subset of original samples may be obtained, and a target sample may be generated based on the feature set of the first feature sample and the second value of the feature. Then, the generative adversarial network may be trained based on the target sample and the second subset of original samples.

[0079] In some embodiments, a feature in feature X is represented by concept C, and when concept C is changed, other features in feature X also need to be changed accordingly. Similar to training the generative adversarial network 509, the generative adversarial network 529 can be trained to learn the mapping of a set of samples of feature X|C=c to a set of samples of feature X|C=c′, and by simultaneously training the generator and discriminator of the generative adversarial network 529, the generator is enabled to generate features that deceive the discriminator as new features of counterfactual samples (for example, Figure 5B Features 526 and 527 in ).

[0080] In some embodiments, when the intervention on the original sample is to change its label, a counterfactual sample corresponding to the original sample can be generated based on the features of the original sample and the changed label. Figure 5C FIG. 5 is a schematic diagram showing a process 540 of generating counterfactual samples by intervening in the features of the original samples according to some embodiments of the present disclosure. Figure 5C As shown, the original sample 502 includes group attribute 504, feature 505, feature 506, feature 506, and label 508. Process 540 intervenes the label 508 of the original sample 502 into label 508 (for example, in the example of predicting whether the user is interested in cars, the value of the label "whether interested in car-related content" can be changed from "yes" to "no"). Figure 3D As described above, when the label of a sample is changed, the characteristics of the sample are not affected. Therefore, process 540 can simply generate counterfactual sample 542 by replacing label 508 of original sample 502 with label 548. The population attributes and characteristics of counterfactual sample 542 are the same as those of original sample 502.

[0081] The various embodiments of generating counterfactual samples described above take into account the causal dependencies between the various concepts of sample data (i.e., group attributes, features, and labels), and in particular, in the case of changing the group attributes and features of the original sample, the rationality of the counterfactual sample can be improved by regenerating the features of the counterfactual sample, thereby improving the accuracy of the assessment of the impact of the original sample on the fairness of the model. In addition, when the generated counterfactual samples are used to repair the original samples, the counterfactual samples with higher rationality can improve the performance and accuracy of the retrained model.

[0082] The above describes the process of generating counterfactual samples and how to calculate the fairness impact value based on the original model, the original samples, and the generated counterfactual samples. After determining the fairness impact of the intervention on the original sample on the original model, the original sample set can be updated based on the determined fairness impact to generate a model with higher fairness.

[0083] In some embodiments, multiple fairness impact values ​​corresponding to multiple original samples can be determined by respectively changing the labels of multiple original samples in the original sample set. Then, multiple target samples with the largest corresponding fairness impact values ​​can be determined from the multiple original samples based on the multiple fairness impact values. Then, an updated sample set can be generated by changing the labels of the multiple target samples, and an updated model can be generated based on the updated sample set.

[0084] In some embodiments, multiple fairness impact values ​​corresponding to multiple original samples can be determined by respectively changing the values ​​of the specified features of multiple original samples in the original sample set. Then, multiple target samples with the largest corresponding fairness impact values ​​can be determined from the multiple original samples based on the multiple fairness impact values. Then, an updated sample set can be generated by removing the multiple target samples from the original sample set, and an updated model can be generated based on the updated sample set.

[0085] In some embodiments, multiple fairness impact values ​​corresponding to multiple original samples can be determined by respectively changing the group attributes of multiple original samples in the original sample set. When the proportion of fairness impact values ​​greater than a predetermined value among the multiple fairness impact values ​​is greater than a predetermined proportion, an updated sample set can be resampled from different groups, and an updated model can be generated based on the updated sample set.

[0086] Through these methods, it is possible to repair the incorrectly labeled labels in the original sample set, identify and remediate fairness poisoning attacks, and discover the unbalanced sampling distribution in the training sample set, so that the population balance of the sample set can be optimized by replacing the original samples with counterfactual samples or recollecting training samples.

[0087] The detailed derivation process of equation (2) is described below. Assume that the risk of model θ can be expressed as And the model trained on the complete training sample set can be expressed as If the weight w i ∈[0, 1] is assigned to each sample Then, by weighting it with some small positive number ∈, the resulting model can be expressed by the following equation (12):

[0088]

[0089] according to The first-order condition of can be obtained as follows:

[0090]

[0091] When ∈→0, using Taylor expansion (and first-order approximation), we can obtain the following equation (14):

[0092]

[0093] according to The first-order condition of Then, by rearranging these terms, we can obtain the following equation (15):

[0094]

[0095] Taking the limit ∈→0 on both sides of equation (15), we can obtain the following equation (16):

[0096]

[0097] Finally, it can be determined as the group by the following equations (17), (18), (19) and (20): The training sample i in the training set is assigned a weight w i Fairness impact:

[0098]

[0099] Figure 6 FIG. 6 is a block diagram of an apparatus 600 for determining the impact of a sample on model fairness according to some embodiments of the present disclosure. Figure 6 As shown, the device 600 includes a counterfactual sample generation module 602, which is configured to generate a counterfactual sample by adjusting the original sample in the original sample set, and the original sample set is used to generate an original model for performing a classification task. The device 600 also includes a fairness index determination module 604, which is configured to determine the fairness index of the original model on the verification sample set. In addition, the device 600 also includes a fairness impact determination module 606, which is configured to determine the fairness impact of the original sample on the original model based on the fairness index, the original sample, and the counterfactual sample.

[0100] It can be understood that by using the apparatus 600 of the present disclosure, at least one of the advantages that can be achieved by the method or process described above can be achieved. For example, the influence of each training sample on the fairness of the model can be determined, helping practitioners to accurately locate those training samples that have the greatest influence on the fairness of the model, so that practitioners can improve the fairness of the model by repairing these training samples.

[0101] Figure 7 FIG. 1 is a block diagram of an electronic device 700 according to some embodiments of the present disclosure. The device 700 may be a device or apparatus described in an embodiment of the present disclosure. Figure 7As shown, the device 700 includes a central processing unit (CPU) and / or a graphics processing unit (GPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 702 or loaded from a storage unit 708 to a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The CPU / GPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704. Although not shown in FIG. Figure 7 As shown in FIG. 7 , device 700 may further include a coprocessor.

[0102] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0103] The various methods or processes described above may be performed by the CPU / GPU 701. For example, in some embodiments, the methods may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the CPU / GPU 701, one or more steps or actions in the methods or processes described above may be performed.

[0104] In some embodiments, the methods and processes described above may be implemented as a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0105] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0106] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0107] The computer program instructions for performing the disclosed operation may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, programming languages ​​including object-oriented programming languages, and conventional procedural programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, executed as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In certain embodiments, the state information of a computer-readable program instruction may be used to customize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), and the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the present disclosure.

[0108] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0109] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0110] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the equipment, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each frame in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the frame can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous frames can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the functions involved. It should also be noted that each frame in the block diagram and / or flow chart, and the combination of frames in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0111] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

[0112] Some example implementations of the present disclosure are listed below.

[0113] Example 1. A method for determining the impact of a sample on model fairness, comprising:

[0114] Generating counterfactual samples by adjusting original samples in an original sample set, wherein the original sample set is used to generate an original model for performing a classification task;

[0115] Determining a fairness indicator of the original model on a validation sample set; and

[0116] Based on the fairness index, the original sample, and the counterfactual sample, the fairness impact of the original sample on the original model is determined.

[0117] Example 2. The method of Example 1, wherein determining the fairness impact comprises:

[0118] Based on the original sample, determine a first loss using a loss function of the original model;

[0119] Determining a second loss using the loss function based on the counterfactual sample; and

[0120] A fairness impact value indicating the fairness impact is determined based on the fairness index, the first loss, and the second loss.

[0121] Example 3. The method of Examples 1-2, wherein determining the fairness impact value comprises:

[0122] Using the loss function, generating a plurality of losses corresponding to a plurality of original samples in the original sample set;

[0123] Based on the plurality of losses, determining a Hessian matrix for the original model; and

[0124] The fairness impact value is determined based on the Hessian matrix, the fairness index, the first loss, and the second loss.

[0125] Example 4. A method according to Examples 1-3, wherein the adjustment to the original sample includes one of the following: changing a population attribute of the original sample, changing a feature of the original sample, changing a label of the original sample, and removing the original sample from the original sample set.

[0126] Example 5. The method of Examples 1-4, wherein generating the counterfactual sample comprises:

[0127] In response to the adjustment to the original sample being to change a population attribute of the original sample, the counterfactual sample is generated based on the changed population attribute and a first feature set of the original sample.

[0128] Example 6. The method of Examples 1-5, wherein generating the counterfactual sample further comprises:

[0129] Based on the changed group attribute and the first feature set of the original sample, using a first generative adversarial network to generate a second feature set of the counterfactual sample; and

[0130] Based on the second feature set of the counterfactual sample, the original model is used to generate a label of the counterfactual sample.

[0131] Example 7. The method according to examples 1-6 further includes:

[0132] Determine a first original sample subset and a second original sample subset from the original sample set, wherein the population attribute of the samples in the first original sample subset has a first value, and the population attribute of the samples in the second original sample subset has a second value;

[0133] Acquire a first population sample from the first original sample subset;

[0134] generating a target sample based on the feature set of the first population sample and the second value of the population attribute; and

[0135] Based on the target sample and the second original sample subset, the first generative adversarial network is generated.

[0136] Example 8. The method of Examples 1-7, wherein generating the counterfactual sample comprises:

[0137] In response to the adjustment to the original sample being to change a feature of the original sample, the counterfactual sample is generated based on the changed feature and a first feature set of the original sample.

[0138] Example 9. The method of claim 1-8, wherein generating the counterfactual sample comprises:

[0139] Based on the changed features and the first feature set of the original sample, using a second generative adversarial network to generate a second feature set of the counterfactual sample; and

[0140] Based on the second feature set of the counterfactual sample, the original model is used to generate a label of the counterfactual sample.

[0141] Example 10. The method according to examples 1-9 further includes:

[0142] Determine a first original sample subset and a second original sample subset from the original sample set, wherein the feature of the samples in the first original sample subset has a first value, and the feature of the samples in the second original sample subset has a second value;

[0143] Acquire a first characteristic sample from the first original sample subset;

[0144] generating a target sample based on the feature set of the first feature sample and the second value of the feature; and

[0145] Based on the target sample and the second original sample subset, the second generative adversarial network is generated.

[0146] Example 11. The method of Examples 1-10, wherein generating the counterfactual sample comprises:

[0147] In response to the adjustment to the original sample being to change the label of the original sample, a counterfactual sample corresponding to the original sample is generated based on the feature set of the original sample and the changed label.

[0148] Example 12. The method of Examples 1-11, further comprising:

[0149] By respectively changing labels of a plurality of original samples in the original sample set, determining a plurality of fairness impact values ​​corresponding to the plurality of original samples;

[0150] Based on the multiple fairness impact values, determining multiple target samples having the largest corresponding fairness impact values ​​from the multiple original samples;

[0151] Generate an updated sample set by changing the labels of the plurality of target samples; and

[0152] An updated model is generated based on the updated sample set.

[0153] Example 13. The method of Examples 1-12 further comprising:

[0154] Determine a plurality of fairness impact values ​​corresponding to the plurality of original samples by respectively changing the values ​​of the specified features of the plurality of original samples in the original sample set;

[0155] Based on the multiple fairness impact values, determining multiple target samples having the largest corresponding fairness impact values ​​from the multiple original samples;

[0156] Generating an updated sample set by removing the plurality of target samples from the original sample set; and

[0157] An updated model is generated based on the updated sample set.

[0158] Example 14. The method of Examples 1-13, further comprising:

[0159] By respectively changing the group attributes of a plurality of original samples in the original sample set, determining a plurality of fairness impact values ​​corresponding to the plurality of original samples;

[0160] In response to a ratio of fairness impact values ​​greater than a predetermined value among the plurality of fairness impact values ​​being greater than a predetermined ratio, resampling an updated sample set from a different population; and

[0161] An updated model is generated based on the updated sample set.

[0162] Example 15. An apparatus for determining the impact of a sample on model fairness, comprising:

[0163] A counterfactual sample generation module, configured to generate counterfactual samples by adjusting original samples in an original sample set, wherein the original sample set is used to generate an original model for performing a classification task;

[0164] A fairness index determination module, configured to determine a fairness index of the original model on a validation sample set; and

[0165] The fairness impact determination module is configured to determine the fairness impact of the original sample on the original model based on the fairness index, the original sample, and the counterfactual sample.

[0166] Example 16. The apparatus of Examples 15-16, wherein determining the fairness impact comprises:

[0167] A first loss determination module is configured to determine a first loss based on the original sample using a loss function of the original model;

[0168] A second loss determination module is configured to determine a second loss based on the counterfactual sample using the loss function; and

[0169] The impact value determination module is configured to determine a fairness impact value indicating the fairness impact based on the fairness index, the first loss, and the second loss.

[0170] Example 17. The apparatus of Examples 15-16, wherein determining the fairness impact value comprises:

[0171] A loss function using module, configured to use the loss function to generate a plurality of losses corresponding to a plurality of original samples in the original sample set;

[0172] A Hessian matrix determination module, configured to determine a Hessian matrix for the original model based on the multiple losses; and

[0173] A Hessian matrix using module is configured to determine the fairness impact value based on the Hessian matrix, the fairness index, the first loss, and the second loss.

[0174] Example 18. An apparatus according to Examples 15-17, wherein the adjustment to the original sample includes one of the following: changing a population attribute of the original sample, changing a feature of the original sample, changing a label of the original sample, and removing the original sample from the original sample set.

[0175] Example 19. The apparatus of Examples 15-18, wherein generating the counterfactual sample comprises:

[0176] The group attribute intervention module is configured to generate the counterfactual sample based on the changed group attribute and the first feature set of the original sample in response to the adjustment to the original sample being to change the group attribute of the original sample.

[0177] Example 20. The apparatus of Examples 15-19, wherein generating the counterfactual sample further comprises:

[0178] A first network using module configured to generate a second feature set of the counterfactual sample using a first generative adversarial network based on the changed group attribute and the first feature set of the original sample; and

[0179] The first label generation module is configured to generate a label of the counterfactual sample based on the second feature set of the counterfactual sample using the original model.

[0180] Example 21. The apparatus of Examples 15-20, further comprising:

[0181] A first subset determination module is configured to determine a first original sample subset and a second original sample subset from the original sample set, wherein the population attribute of the samples in the first original sample subset has a first value, and the population attribute of the samples in the second original sample subset has a second value;

[0182] A group sample acquisition module, configured to acquire a first group sample from the first original sample subset;

[0183] A first target generating module configured to generate a target sample based on the feature set of the first group sample and the second value of the group attribute; and

[0184] The first network generation module is configured to generate the first generative adversarial network based on the target sample and the second original sample subset.

[0185] Example 22. The apparatus of Examples 15-21, wherein generating the counterfactual sample comprises:

[0186] The sample feature intervention module is configured to generate the counterfactual sample based on the changed feature and a first feature set of the original sample in response to the adjustment to the original sample being to change the feature of the original sample.

[0187] Example 23. The apparatus of claim 15-22, wherein generating the counterfactual sample comprises:

[0188] a second network using module configured to generate a second feature set of the counterfactual sample using a second generative adversarial network based on the changed features and the first feature set of the original sample; and

[0189] The second label generation module is configured to generate a label of the counterfactual sample based on the second feature set of the counterfactual sample and using the original model.

[0190] Example 24. The apparatus of Examples 15-23, further comprising:

[0191] A second subset determination module is configured to determine a first original sample subset and a second original sample subset from the original sample set, wherein the feature of the samples in the first original sample subset has a first value, and the feature of the samples in the second original sample subset has a second value;

[0192] A feature sample acquisition module, configured to acquire a first feature sample from the first original sample subset;

[0193] A second target generation module is configured to generate a target sample based on the feature set of the first feature sample and the second value of the feature; and

[0194] The second network generation module is configured to generate the second generative adversarial network based on the target sample and the second original sample subset.

[0195] Example 25. The apparatus of Examples 15-24, wherein generating the counterfactual sample comprises:

[0196] The sample label intervention module is configured to generate a counterfactual sample corresponding to the original sample based on the feature set of the original sample and the changed label in response to the adjustment to the original sample being to change the label of the original sample.

[0197] Example 26. The apparatus of Examples 15-25, further comprising:

[0198] a label intervention application module, configured to determine a plurality of fairness impact values ​​corresponding to the plurality of original samples by respectively changing labels of the plurality of original samples in the original sample set;

[0199] A first impact value comparison module is configured to determine, based on the plurality of fairness impact values, a plurality of target samples having the largest corresponding fairness impact values ​​from the plurality of original samples;

[0200] A first update sample generation module is configured to generate an update sample set by changing labels of the plurality of target samples; and

[0201] The first update module generation module is configured to generate an update model based on the update sample set.

[0202] Example 27. The apparatus of Examples 15-26, further comprising:

[0203] a feature intervention application module configured to determine a plurality of fairness impact values ​​corresponding to the plurality of original samples in the original sample set by respectively changing values ​​of designated features of the plurality of original samples;

[0204] A second impact value comparison module is configured to determine, based on the plurality of fairness impact values, a plurality of target samples having the largest corresponding fairness impact values ​​from the plurality of original samples;

[0205] a second updated sample generating module, configured to generate an updated sample set by removing the plurality of target samples from the original sample set; and

[0206] The second update model generation module is configured to generate an update model based on the update sample set.

[0207] Example 28. The apparatus of Examples 15-27, further comprising:

[0208] A group attribute application module is configured to determine a plurality of fairness impact values ​​corresponding to the plurality of original samples by respectively changing the group attributes of the plurality of original samples in the original sample set;

[0209] A third update sample generating module is configured to resample an update sample set from a different group in response to a ratio of fairness impact values ​​greater than a predetermined value among the plurality of fairness impact values ​​being greater than a predetermined ratio; and

[0210] The third update module generation module is configured to generate an update model based on the update sample set.

[0211] Example 29. An electronic device comprising:

[0212] Processor; and

[0213] A memory coupled to the processor, the memory having instructions stored therein, the instructions causing the electronic device to perform actions when executed by the processor, the actions comprising:

[0214] Generating counterfactual samples by adjusting original samples in an original sample set, wherein the original sample set is used to generate an original model for performing a classification task;

[0215] Determining a fairness indicator of the original model on a validation sample set; and

[0216] Based on the fairness index, the original sample, and the counterfactual sample, the fairness impact of the original sample on the original model is determined.

[0217] Example 30. The apparatus of Example 29, wherein determining the fairness impact comprises:

[0218] Based on the original sample, determine a first loss using a loss function of the original model;

[0219] Based on the counterfactual sample, determining a second loss using the loss function; and

[0220] A fairness impact value indicating the fairness impact is determined based on the fairness index, the first loss, and the second loss.

[0221] Example 31. The apparatus of Examples 29-30, wherein determining the fairness impact value comprises:

[0222] Using the loss function, generating a plurality of losses corresponding to a plurality of original samples in the original sample set;

[0223] Based on the plurality of losses, determining a Hessian matrix for the original model; and

[0224] The fairness impact value is determined based on the Hessian matrix, the fairness index, the first loss, and the second loss.

[0225] Example 32. An apparatus according to Examples 29-31, wherein the adjustment to the original sample includes one of the following: changing a population attribute of the original sample, changing a feature of the original sample, changing a label of the original sample, and removing the original sample from the original sample set.

[0226] Example 33. The apparatus of Examples 29-32, wherein generating the counterfactual sample comprises:

[0227] In response to the adjustment to the original sample being to change a population attribute of the original sample, the counterfactual sample is generated based on the changed population attribute and a first feature set of the original sample.

[0228] Example 34. The apparatus of Examples 29-33, wherein generating the counterfactual sample further comprises:

[0229] Based on the changed group attribute and the first feature set of the original sample, using a first generative adversarial network to generate a second feature set of the counterfactual sample; and

[0230] Based on the second feature set of the counterfactual sample, the original model is used to generate a label of the counterfactual sample.

[0231] Example 35. The apparatus of Examples 29-34, further comprising:

[0232] Determine a first original sample subset and a second original sample subset from the original sample set, wherein the population attribute of the samples in the first original sample subset has a first value, and the population attribute of the samples in the second original sample subset has a second value;

[0233] Acquire a first population sample from the first original sample subset;

[0234] generating a target sample based on the feature set of the first population sample and the second value of the population attribute; and

[0235] Based on the target sample and the second original sample subset, the first generative adversarial network is generated.

[0236] Example 36. The apparatus of Examples 29-35, wherein generating the counterfactual sample comprises:

[0237] In response to the adjustment to the original sample being to change a feature of the original sample, the counterfactual sample is generated based on the changed feature and a first feature set of the original sample.

[0238] Example 37. The apparatus of claims 29-36, wherein generating the counterfactual sample comprises:

[0239] Based on the changed features and the first feature set of the original sample, using a second generative adversarial network to generate a second feature set of the counterfactual sample; and

[0240] Based on the second feature set of the counterfactual sample, the original model is used to generate a label of the counterfactual sample.

[0241] Example 38. The apparatus of Examples 29-37, further comprising:

[0242] Determine a first original sample subset and a second original sample subset from the original sample set, wherein the feature of the samples in the first original sample subset has a first value, and the feature of the samples in the second original sample subset has a second value;

[0243] Acquire a first characteristic sample from the first original sample subset;

[0244] generating a target sample based on the feature set of the first feature sample and the second value of the feature; and

[0245] Based on the target sample and the second original sample subset, the second generative adversarial network is generated.

[0246] Example 39. The apparatus of Examples 29-38, wherein generating the counterfactual sample comprises:

[0247] In response to the adjustment to the original sample being to change the label of the original sample, a counterfactual sample corresponding to the original sample is generated based on the feature set of the original sample and the changed label.

[0248] Example 40. The apparatus of Examples 29-39, further comprising:

[0249] By respectively changing labels of a plurality of original samples in the original sample set, determining a plurality of fairness impact values ​​corresponding to the plurality of original samples;

[0250] Based on the multiple fairness impact values, determining multiple target samples having the largest corresponding fairness impact values ​​from the multiple original samples;

[0251] Generate an updated sample set by changing the labels of the plurality of target samples; and

[0252] An updated model is generated based on the updated sample set.

[0253] Example 41. The apparatus of Examples 29-40, further comprising:

[0254] Determine a plurality of fairness impact values ​​corresponding to the plurality of original samples by respectively changing the values ​​of the specified features of the plurality of original samples in the original sample set;

[0255] Based on the multiple fairness impact values, determining multiple target samples having the largest corresponding fairness impact values ​​from the multiple original samples;

[0256] Generating an updated sample set by removing the plurality of target samples from the original sample set; and

[0257] An updated model is generated based on the updated sample set.

[0258] Example 42. The apparatus of Examples 29-41, further comprising:

[0259] By respectively changing the group attributes of a plurality of original samples in the original sample set, determining a plurality of fairness impact values ​​corresponding to the plurality of original samples;

[0260] In response to a ratio of fairness impact values ​​greater than a predetermined value among the plurality of fairness impact values ​​being greater than a predetermined ratio, resampling an updated sample set from a different population; and

[0261] An updated model is generated based on the updated sample set.

[0262] Although the disclosure has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Instead, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for determining the impact of a sample on model fairness, comprising: Generating counterfactual samples by adjusting original samples in an original sample set, wherein the original sample set is used to generate an original model for performing a classification task; Determining a fairness indicator of the original model on a validation sample set; as well as Based on the fairness index, the original sample, and the counterfactual sample, the fairness impact of the original sample on the original model is determined.

2. The method of claim 1 , wherein determining the fairness impact comprises: Based on the original sample, determine a first loss using a loss function of the original model; Based on the counterfactual sample, determining a second loss using the loss function; as well as A fairness impact value indicating the fairness impact is determined based on the fairness index, the first loss, and the second loss.

3. The method according to claim 2, wherein determining the fairness impact value comprises: Using the loss function to generate a plurality of losses corresponding to a plurality of original samples in the original sample set; Based on the multiple losses, determining a Hessian matrix for the original model; as well as The fairness impact value is determined based on the Hessian matrix, the fairness index, the first loss, and the second loss.

4. The method according to claim 1, wherein the adjustment to the original sample comprises one of the following: changing a population attribute of the original sample, changing a feature of the original sample, changing a label of the original sample, or removing the original sample from the original sample set.

5. The method of claim 4, wherein generating the counterfactual sample comprises: In response to the adjustment to the original sample being to change a population attribute of the original sample, the counterfactual sample is generated based on the changed population attribute and a first feature set of the original sample.

6. The method according to claim 5, wherein generating the counterfactual sample further comprises: Based on the changed group attribute and the first feature set of the original sample, using a first generative adversarial network to generate a second feature set of the counterfactual sample; as well as Based on the second feature set of the counterfactual sample, the original model is used to generate a label of the counterfactual sample.

7. The method according to claim 6, further comprising: Determine a first original sample subset and a second original sample subset from the original sample set, wherein the population attribute of the samples in the first original sample subset has a first value, and the population attribute of the samples in the second original sample subset has a second value; Acquire a first population sample from the first original sample subset; generating a target sample based on the feature set of the first group sample and the second value of the group attribute; as well as Based on the target sample and the second original sample subset, the first generative adversarial network is generated.

8. The method of claim 4, wherein generating the counterfactual sample comprises: In response to the adjustment to the original sample being to change a feature of the original sample, the counterfactual sample is generated based on the changed feature and a first feature set of the original sample.

9. The method of claim 8, wherein generating the counterfactual sample comprises: Based on the changed features and the first feature set of the original sample, generating a second feature set of the counterfactual sample using a second generative adversarial network; as well as Based on the second feature set of the counterfactual sample, the original model is used to generate a label of the counterfactual sample.

10. The method according to claim 9, further comprising: Determine a first original sample subset and a second original sample subset from the original sample set, wherein the feature of the samples in the first original sample subset has a first value, and the feature of the samples in the second original sample subset has a second value; Acquire a first characteristic sample from the first original sample subset; Generate a target sample based on the feature set of the first feature sample and the second value of the feature; as well as Based on the target sample and the second original sample subset, the second generative adversarial network is generated.

11. The method of claim 4, wherein generating the counterfactual sample comprises: In response to the adjustment to the original sample being to change the label of the original sample, a counterfactual sample corresponding to the original sample is generated based on the feature set of the original sample and the changed label.

12. The method according to claim 1, further comprising: By respectively changing labels of a plurality of original samples in the original sample set, determining a plurality of fairness impact values ​​corresponding to the plurality of original samples; Based on the multiple fairness impact values, determining multiple target samples having the largest corresponding fairness impact values ​​from the multiple original samples; Generate an updated sample set by changing the labels of the multiple target samples; as well as An updated model is generated based on the updated sample set.

13. The method according to claim 1, further comprising: Determine a plurality of fairness impact values ​​corresponding to the plurality of original samples by respectively changing the values ​​of the specified features of the plurality of original samples in the original sample set; Based on the multiple fairness impact values, determining multiple target samples having the largest corresponding fairness impact values ​​from the multiple original samples; Generating an updated sample set by removing the plurality of target samples from the original sample set; as well as An updated model is generated based on the updated sample set.

14. The method according to claim 1, further comprising: By respectively changing the group attributes of a plurality of original samples in the original sample set, determining a plurality of fairness impact values ​​corresponding to the plurality of original samples; In response to a ratio of fairness impact values ​​greater than a predetermined value among the plurality of fairness impact values ​​being greater than a predetermined ratio, resampling an updated sample set from a different group; as well as An updated model is generated based on the updated sample set.

15. A device for determining the impact of a sample on model fairness, comprising: A counterfactual sample generation module, configured to generate counterfactual samples by adjusting original samples in an original sample set, wherein the original sample set is used to generate an original model for performing a classification task; A fairness index determination module, configured to determine a fairness index of the original model on a validation sample set; as well as The fairness impact determination module is configured to determine the fairness impact of the original sample on the original model based on the fairness index, the original sample, and the counterfactual sample.

16. An electronic device, comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, wherein when the instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 14.

17. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions are executed by a processor to implement the method according to any one of claims 1 to 14.

Images