Information processing device, information processing method, computer program, and image sensor

Abstract

According to the invention, deviation and visual deviation are removed. This information processing device is equipped with: an acquisition unit that acquires, using XAI, information on a deviation included in data or a model, the deviation being caused by a sensitive attribute; and a processing unit that performs processing for mitigating a deviation in the data or the model on the basis of the acquired information. The acquisition unit is provided with: a prediction model that predicts sensitive attributes from non-sensitive attribute data items in input data consisting of a plurality of data items; and a contribution degree calculation unit that calculates a contribution degree of each non-sensitive attribute data item on the basis of a determination basis of a prediction model obtained using the XAI.

Description

Technical Field

[0001] The technology disclosed in this specification (hereinafter, “the present disclosure”) relates to an information processing device, an information processing method, a computer program, and an image sensor that perform processing related to training of artificial intelligence (AI). Background Art

[0002] The evolution of artificial intelligence (AI) has been remarkable, and has achieved recognition, identification, prediction, and the like beyond human capabilities, and has advanced its use in various fields. On the other hand, ethical issues with AI have arisen, and it is a social issue that unfair decisions are made through machine learning based on race, gender, and the like. The causes of this unfairness in AI include bias caused by the datasets used for training and bias during training. Therefore, the challenge is to mitigate bias in the dataset collection stage and make fair decisions by learning from the collected datasets. However, in reality, it is difficult to collect a complete dataset that ensures fairness with respect to race and gender. This is because the random collection of data causes some biased data distribution, which affects machine learning as data bias.

[0003] For example, a system for managing a model trained by machine learning is proposed, in which fairness is ensured by rewriting the content of a prediction request, and a dataset is enhanced and retrained by fixing the values ​​of the rewritten items and changing other values ​​(see Patent Document 1).

[0004] In addition, there is a method called FairGAN (see non-patent document 1), which generates data in which the distribution of training data does not change according to the presence or absence of sensitive attributes by using a generative adversary network (GAN). In addition to a generator G for newly generating fair training data from noise, the FairGAN model also includes two classifiers: a classifier D1 for identifying whether the given data is original data or data generated by the generator G; and a classifier D2 for identifying whether the generated data has sensitive attributes. The generator G is trained to minimize the performance of each of the classifiers D1 and D2. The data generated by the trained generator G has a structure similar to the original training data and is independent of the sensitive attributes.

[0005] Reference List

[0006] Patent Literature

[0007] Patent Document 1: JP 2021-12593A

[0008] Non-patent literature

[0009] Non-Patent Literature 1: Xu et al., “FairGAN: Fairness-aware Generative Adversarial Networks,” IEEE BigData 2018.

[0010] Non-patent literature 2: Lundberg, Scott M., and Su-In Lee. "A unified approach to interpreting model predictions." Advances in neural information processing systems. 12017.

[0011] Non-Patent Literature 3: “Why Should I Trust You?”: Explaining the Predictions of Any Classifier<https: / / arxiv.org / abs / 1602.04938>

[0012] Non-patent document 4: Residuals and Influence in Regression, Cook, RD and Weisberg, S<https: / / conservancy.umn.edu / handle / 11299 / 37076> Summary of the Invention

[0013] Problems to be solved by the present invention

[0014] It is desirable to provide an information processing apparatus, an information processing method, a computer program, and an image sensor that perform processing for ensuring fairness in artificial intelligence (AI).

[0015] Solution to the problem

[0016] The present disclosure is made in view of the above-mentioned problems, and a first aspect of the present disclosure is an information processing device including:

[0017] an acquisition unit that acquires information about bias caused by sensitive attributes included in data or models by using explainable artificial intelligence (XAI); and

[0018] A processing unit performs processing to reduce bias of the data or model based on the acquired information.

[0019] In an information processing device according to the first aspect, an acquisition unit includes: a preprocessing unit that removes data items related to sensitive attributes from input data including multiple data items; a prediction unit that predicts sensitive attributes based on data items of non-sensitive attributes in the input data; and a contribution calculation unit that calculates the contribution of each data item of non-sensitive attributes in the input data to the prediction result of the prediction unit. The prediction unit predicts sensitive attributes from the data items of non-sensitive attributes in the input data by using a prediction model with sensitive attributes as target variables. Then, the contribution calculation unit calculates the contribution of each data item of non-sensitive attributes based on the determination basis of the prediction model obtained by using XAI. Therefore, the processing unit can modify the correct answer label corresponding to the input data based on the contribution calculation result and generate a data set that reduces the bias caused by sensitive attributes. The processing unit can modify the correct answer label for each sensitive attribute value of the input data, the input data including data items with higher contribution among non-sensitive attributes, and generate a data set in which the bias caused by sensitive attributes is reduced.

[0020] Alternatively, in the information processing device of the first aspect, the acquisition unit further includes an influence function calculation unit that calculates an influence function of the sensitive attribute based on the result of determining the input data using the trained model. For example, the input data is an image captured by an image sensor, and the trained model is a model trained to detect people from the image. In this case, the influence function calculation unit calculates the influence function of the sensitive attribute when the trained model detects a person from the image. The processing unit can then retrain the model by adding images of the sensitive attribute with low influence function values, thereby reducing model bias.

[0021] In addition, the second aspect of the present disclosure is

[0022] Information processing methods, including:

[0023] By using explainable artificial intelligence (XAI) to obtain information about biases caused by sensitive attributes included in data or models; and

[0024] Based on the acquired information, a process of mitigating bias in the data or model is performed.

[0025] Furthermore, a third aspect of the present disclosure is:

[0026] A computer program described in a computer-readable format, which causes a computer to:

[0027] an acquisition unit that acquires information about bias caused by sensitive attributes included in data or models by using explainable artificial intelligence (XAI); and

[0028] A processing unit performs processing to reduce bias of the data or model based on the acquired information.

[0029] The computer program according to the third aspect of the present disclosure is obtained by defining a computer program described in a computer-readable format in a manner that implements a predetermined process on a computer. The computer program can be provided to a target computer capable of executing various program codes through a storage medium or communication medium provided in a computer-readable form (for example, a storage medium such as an optical disc, a magnetic disk, or a semiconductor memory, or a communication medium such as a network). Then, the computer program according to the third aspect of the present disclosure installed in a computer using any one of the media applies a cooperative action on the computer, so that an operational effect similar to that of the device according to the first aspect of the present disclosure can be obtained.

[0030] Furthermore, a fourth aspect of the present disclosure is:

[0031] Image sensors, including:

[0032] Multilayer semiconductor chips,

[0033] wherein a sensor unit including a pixel array and capturing an image is mounted on a first layer, and a memory unit and a logic unit are mounted on a second layer and subsequent layers, the memory unit stores the captured image of the sensor unit, and the logic unit controls driving of the sensor unit and processes the captured image of the sensor unit, and

[0034] The logic unit includes: an inference unit that performs inference on a captured image by using a trained model; and a determination basis calculation unit that calculates a determination basis from a result of the inference performed by the trained model.

[0035] The inference unit detects a person from a captured image by using the trained model. In addition, the determination basis calculation unit calculates an influence function of a sensitive attribute when a person is detected from an image by the trained model.

[0036] Effects of the present invention

[0037] According to the present disclosure, an information processing device, an information processing method, a computer program, and an image sensor that visualizes a bias included in AI training data or AI by using XAI and removes the bias may be provided.

[0038] It should be noted that the effects described in this specification are merely examples, and the effects brought about by the present disclosure are not limited thereto. In addition, in addition to the above-described effects, the present disclosure may further provide additional effects.

[0039] Other objects, features, and advantages of the present disclosure will become apparent through more detailed description based on embodiments to be described later and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a diagram illustrating a functional configuration example of the training system 100 .

[0041] Figure 2 is a flow chart illustrating an exemplary process performed in the training system 100 .

[0042] Figure 3 is a diagram showing a functional configuration of the data deviation alleviating unit 121 .

[0043] Figure 4 is a diagram illustrating an example of demographic parity.

[0044] Figure 5 is a diagram illustrating an example of balanced odds.

[0045] Figure 6 is a diagram illustrating an example of equal opportunity.

[0046] Figure 7 is a diagram illustrating the distribution transformation on the dataset [Y, X, S] with bias mitigation.

[0047] Figure 8 2000 is a diagram illustrating a configuration example of the information processing apparatus 2000 .

[0048] Figure 9 is a diagram showing a functional configuration of the imaging device 800 .

[0049] Figure 10 is a diagram illustrating a hardware implementation example of the image sensor 1000 .

[0050] Figure 11 1100 is a diagram illustrating a hardware implementation example of the image sensor 1100 .

[0051] Figure 12 1 is a diagram illustrating an example in which the image sensor 1000 (or 1100 ) is formed as a semiconductor chip 1200 having a two-layer structure in which two layers are stacked.

[0052] Figure 13 1 is a diagram illustrating an example in which the image sensor 1000 (or 1100 ) is formed as a semiconductor chip 1300 having a three-layer structure in which three layers are stacked.

[0053] Figure 14 is a diagram illustrating an operation example of the training system 100 .

[0054] Figure 15 It is shown by Figure 14 is a diagram of an example of a communication sequence performed by a client-server system shown in .

[0055] Figure 16 is a diagram illustrating an operation example of a digital camera mounted with an image sensor including a multilayer semiconductor chip including a logic unit.

[0056] Figure 17 is a diagram illustrating an example of Shapley values ​​calculated in the case of predicting a sensitive attribute S from a non-sensitive attribute X. DETAILED DESCRIPTION

[0057] Hereinafter, embodiments of the present disclosure will be described in the following order with reference to the accompanying drawings.

[0058] A. Overview

[0059] B. Configuration and Operation of the Training System

[0060] B-1. System Configuration

[0061] B-2. System Operation

[0062] C. Bias Mitigation Mechanism

[0063] C-1. Bias Mitigation Mechanism

[0064] C-2. Fairness

[0065] C-3. Explainable Data Bias Mitigation Methods

[0066] C-4. Configuration of Information Processing Device

[0067] D. Second Implementation

[0068] D-1. Configuration of imaging device

[0069] D-2. Image Sensor Configuration

[0070] D-3. System Configuration and Operation

[0071] A. Overview

[0072] Despite the remarkable evolution of artificial intelligence (AI), ethical issues surrounding AI are already emerging. Training AI using datasets with biased distributions can introduce bias and potentially produce unfair results. Conversely, AI trained using unbiased and fair datasets can produce fair results independent of sensitive attributes while maintaining predictive or inference performance. To achieve this, collecting unbiased and fair datasets is a challenge.

[0073] Various methods have been proposed for achieving fairness in AI machine learning that do not rely on sensitive attributes. However, because the AI ​​methods themselves are black boxes, it is difficult for humans to intuitively understand the internal processing used to determine fairness. Furthermore, there are certain limitations on the collection of diverse data.

[0074] Therefore, according to the present disclosure, a bias mitigation method using explainable AI (XAI) is proposed. According to the present disclosure, information related to bias caused by sensitive attributes included in input data can be obtained by using XAI, and bias mitigation processing can be further performed based on the obtained information.

[0075] In a first embodiment of the present invention (described later), sensitive attribute data in input data is predicted based on non-sensitive attribute data in the input data using a prediction model, a determination basis of the prediction model is calculated using XAI, and the correct answer label corresponding to the input data is modified based on the calculation result of XAI, thereby mitigating the bias caused by sensitive attributes, which is potential in the data set used for training.

[0076] In addition, in a second embodiment of the present disclosure (described later), an influence function for the inference result of the trained model is calculated for each data item of a sensitive attribute in the input data by using XAI. For example, in the case where a person detection is performed on an image sensor having an AI function and from an image captured by the image sensor through AI, an influence function for each data item of a sensitive attribute of the detection result is calculated. Then, in the case where a data item of a sensitive attribute with a low influence score is found in the person detection, the deviation of the AI ​​caused by the sensitive attribute is mitigated by adding an image corresponding to the data item and retraining the AI.

[0077] Thus, according to the present disclosure, the potential for bias in datasets where sensitive attributes (such as race, age, and gender) are unknown can be eliminated, and how bias mitigation is achieved in a human-understandable manner can be explained.

[0078] B. Configuration and Operation of the Training System

[0079] In a first embodiment of the present invention, a prediction model is used to predict sensitive attributes from input data lacking sensitive attribute data, a determination basis for the prediction model is calculated using XAI, and the correct answer label corresponding to the input data is modified based on the calculation result of XAI, thereby mitigating data bias caused by sensitive attributes, which is latent in the dataset used for training. Furthermore, in a second embodiment of the present disclosure, an influence function for the inference result of the trained model is calculated for each sensitive attribute data item in the input data using XAI, and retraining is performed based on each influence score to mitigate model bias. In this Section B, the configuration and operation of the training system applied to each embodiment of the present disclosure will be described.

[0080] B-1. System Configuration

[0081] Figure 1 An example of a functional configuration of a training system 100 to which the first embodiment of the present disclosure is mainly applied is shown. Although the training system 100 shown is used by being installed on, for example, an edge device, some or all of the functions of the training system 100 may be constructed on the cloud or an arithmetic device capable of large-scale computing. Hereinafter, the training system 100 trains a model (a neural network model, etc.) for performing image recognition (e.g., face detection, facial recognition, person / object detection, or posture estimation), and may also train a model for performing inference, for example, image-based personnel recruitment, recidivism rate, and loan approval. However, the present disclosure is not limited thereto, and the training system 100 may train models for inference in various fields.

[0082] The training system 100 shown includes a dataset holding unit 101, a model training unit 102, a model parameter holding unit 103, an inference unit 111, a data input unit 112, and an input data processing unit 113. The dataset holding unit 101, the model training unit 102, and the model parameter holding unit 103 operate during the model training phase, while the inference unit 111, the data input unit 112, and the input data processing unit 113 operate during the inference phase, which utilizes the trained model. Although the training system 100 is used by being installed on, for example, an edge device, some or all of the functions of the training system 100 can be configured on the cloud or an arithmetic device capable of large-scale computing. For example, the model training unit 102, which processes a large amount of data and requires high computing power, can be located in a server on the cloud, and the inference unit 111, which utilizes the trained model, can be located in a digital camera, a multi-functional information terminal such as a smartphone or tablet, or an edge device such as a personal computer.

[0083] In addition, in this embodiment, the training system 100 also includes a dataset collection unit 130 and a data bias mitigation unit 121. The dataset collection unit 130 provides a dataset for model training, and the data bias mitigation unit 121 mitigates the potential bias in the dataset used for model training in the model training unit 102.

[0084] In addition, the training system 100 may further include: a model bias determination unit 123, which determines bias when the inference unit 111 performs inference by using the trained model; and a model bias mitigation unit 124, which performs processing for mitigating the bias of the trained model based on the result of determining the bias of the model.

[0085] The dataset collection unit 130 collects datasets used for model training by the model training unit 102. The dataset collection unit 130 collects datasets via a wide area network such as the Internet and accumulates the datasets in the dataset storage unit 101. The dataset collection unit 130 may collect datasets from a service that provides datasets for training.

[0086] The dataset basically includes a combination of input data input to the model to be trained and observations of the input data. The input data includes sensitive attribute data S and attribute data X of non-sensitive attributes (i.e., non-sensitive attribute data X). The sensitive attribute data S basically includes gender, race, and age. However, depending on the application, any of gender, race, and age can be omitted, or part of the non-sensitive attribute data can be regarded as sensitive attribute data. In addition, in the case of training the model, the observation value corresponds to the correct answer label Y. In the following, the dataset is described as [Y, X, S].

[0087] In addition, in the case where the model training unit 102 trains a model for image classification such as person detection or object detection, the dataset collection unit 130 collects a dataset including a combination of captured images from a large number of digital cameras and a correct answer label (subject of the captured image) via a wide area network such as the Internet, and accumulates the dataset into the dataset holding unit 101.

[0088] The dataset holding unit 101 accumulates the dataset to be used for model training by the model training unit 102. The dataset holding unit 101 accumulates the dataset provided by the dataset collecting unit 130, and may also accumulate the dataset obtained from other sources. 0 The dataset [Y 0 , X, S] is also accumulated in the data set holding unit 101, where the data correctly answers the label Y 0In the example, the deviation caused by the sensitive attribute data S in the non-sensitive attribute data X is reduced (i.e., corrected) by the data deviation reduction unit 121 (described later). In the case where the model training unit 102 performs deep learning, a large amount of necessary data sets are accumulated in the data set holding unit 101. The data set holding unit 101 is implemented by a large-capacity storage device, for example, a data server.

[0089] The model training unit 102 reads the data set from the data set holding unit 101 in sequence, performs model training, and updates the model parameters. The model to be trained is, for example, a neural network including a combination of neurons, and can also be a model using a type such as support vector regression or Gaussian process regression. The neural network model has a multi-layer structure, which includes an input layer that receives data (illustrative variables) such as images, an output layer that outputs a label (target variable) as an inference result of the input data, and one or more intermediate layers (or hidden layers) between the input layer and the output layer. Each layer includes a plurality of nodes corresponding to neurons. The connections between nodes across the layers have weights, and when data is passed from layer to layer, the value of the data input to the input layer is transformed.

[0090] For example, the model training unit 102 calculates a loss function L, which is defined based on the error between the output Y^ of the model from the input data [X, S] and the known correct answer label Y corresponding to the input data, and performs model training while updating the model parameters by backpropagation in a manner that minimizes the loss function L. The model training unit 102 then stores the model parameters obtained as a result of the training in the model parameter holding unit 103. Model parameters are variable elements that define the model, and are, for example, connection weight coefficients of neurons to be given to the neural network model, etc. It should be noted that in the formula, the estimated value of the Y output from the model is expressed by adding the accent symbol "^" to the letter "Y". In this specification, the estimated value of Y is expressed as "Y^" by connecting the accent symbol "^" immediately after the character "Y".

[0091] Note that since the model training process requires a large amount of calculation, the model training unit 102 can perform training by using an information processing device equipped with an arithmetic circuit (such as a multi-core central unit (CPU), a graphics processing unit (GPU), or a general-purpose computing on a graphics processing unit (GPGPU)), or can use multiple computing nodes to perform distributed training.

[0092] The inference unit 111, the data input unit 112, and the input data processing unit 113 implement the inference stage by using the trained model. The functional modules 111 to 113 are installed on an edge device using the trained model, for example. The data input unit 112 inputs data inferred from the outside. The data input unit 112 can input data via a console or can input sensor information acquired by a sensor included in the edge device. The input data processing unit 113 performs reshaping processing in such a manner that the data input from the data input unit 112 has a data format that can be input to the trained model, and inputs the data to the inference unit 111. The input data of the inference unit 111 is [X, S] including non-sensitive attribute data X and sensitive attribute data S. The inference unit 111 outputs a predicted value Y^ predicted by the trained model from the input data [X, S]. The trained model is a model in which the model parameters read from the model parameter holding unit 103 are set.

[0093] The data deviation mitigation unit 121 sequentially extracts the data set [Y, X, S] accumulated in the data set holding unit 101, corrects the correct answer label Y corresponding to the input data [X, S] in a manner that reduces the deviation potentially caused by the sensitive attribute data S in the non-sensitive attribute data X, and sends the corrected correct answer label Y to the non-sensitive attribute data X. o The dataset [Y o , X, S] is written back to the data set holding unit 101. In the first embodiment of the present invention, the data deviation mitigation unit 121 predicts the sensitive attribute S^ from the input data X without sensitive attribute data by using a prediction model and calculates the basis for determining the sensitive attribute S^ by the prediction model by using XAI. Specifically, for each data item of the non-sensitive attribute in the input data, the Shapley value according to the cooperative game theory is calculated as the contribution to the prediction. Then, the deviation caused by the sensitive attributes in the potential data set [Y, X, S] is mitigated by modifying the correct answer label for each sensitive attribute value of the input data, which includes data items with higher Shapley values ​​in the non-sensitive attributes. The details of the data deviation mitigation process will be described in detail in the following section C.

[0094] Note that various settings in the data bias mitigation process in the data bias mitigation unit 121 can be set by user operations via a user interface (UI). Furthermore, the data bias mitigation unit 121 can use the UI to visualize how the bias of the input data is mitigated in a way that is understandable to humans. For example, the data bias mitigation unit 121 can provide feedback to the user on the Shapley value calculated for each data item of a non-sensitive attribute in the input data, or information about the data bias mitigation process based on the Shapley value.

[0095] The model bias determination unit 123 uses the trained model to analyze the inference result performed by the inference unit 111 and determines whether the fairness of the model is ensured, that is, the model bias. In a second embodiment (described later), XAI is used to calculate the influence function of the data item of each sensitive attribute in the input data of the result of the inference performed by the trained model (that is, the inference unit 111). Subsequently, the model bias determination unit 123 detects whether there is a data item with a low influence score in the inference result of the trained model. The model bias determination unit 123 can then determine that a negative bias is applied to the data item of the sensitive attribute with a low influence score, that is, the fairness of the model cannot be ensured.

[0096] The model bias mitigation unit 124 performs a process for mitigating model bias based on the model bias determination result. The model bias mitigation unit 124 may perform a model bias mitigation algorithm such as transfer learning, fine-tuning, and incremental learning, and the model training unit 102 may mitigate the trained model bias.

[0097] In the second embodiment of the present disclosure, the model deviation mitigation unit 124 adds input data corresponding to the data items of the sensitive attributes that the model deviation determination unit 123 has determined to have low influence scores for the inference of the trained model, and instructs the model training unit 102 to retrain the model by using the added input data. As a result, the model is retrained so that even the influence of the sensitive attributes with low influence scores is increased, and the model deviation caused by the sensitive attributes can be mitigated.

[0098] Note that various settings in the model bias mitigation process can be set by user operations via the UI. Furthermore, the model bias mitigation unit 124 can visualize how to mitigate model bias by performing presentation using the UI so that humans can understand.

[0099] B-2. System Operation

[0100] Figure 2 A schematic process performed in the training system 100 is shown in the form of a flowchart. The illustrated process is classified into a training phase in which model training is performed and an inference phase in which inference is performed using the trained model, and has a main feature in that the process includes a process for mitigating bias in training data (i.e., "data bias" in the training phase) and a process for mitigating bias in inference of the trained model (i.e., "model bias" in the inference phase).

[0101] First, the training phase will be described. The dataset collection unit 130 collects a dataset to be used by the model training unit 102 for model training (step S201). The dataset collection unit 130 collects the dataset via a wide area network such as the Internet and accumulates the dataset in the dataset holding unit 101. However, the source from which the dataset collection unit 130 collects training data is not particularly limited.

[0102] The data bias mitigation unit 121 mitigates the potential for bias in the datasets used for model training in the model training unit 102, for the datasets accumulated in the dataset holding unit 101. Specifically, the data bias mitigation unit 121 repeats the following series of processes: reading the dataset [Y, X, S] from the dataset holding unit 101 (step S202), predicting the sensitive attribute S^ from the non-sensitive attribute data X using the prediction model (step S203), calculating the basis for determining the sensitive attribute S^ using the prediction model using XAI (step S204), and modifying the correct answer label Y corresponding to the input data X based on the calculation result (step S205), as a modification process for mitigating bias in each dataset accumulated in the dataset holding unit 101. In step S204, a Shapley value is calculated as the contribution to the prediction of each data item of the non-sensitive attribute data X in the input data. Then, in step S205 , for each sensitive attribute value of the input data including data items with higher Shapley values ​​in the non-sensitive attribute data X, the correct answer label Y is modified, thereby mitigating the bias caused by potential sensitive attributes in the dataset used for training.

[0103] Then, the model training unit 102 trains the model to be trained by using a data set that reduces data bias, and updates the model parameters (step S206). For example, in the case where a convolutional neural network is to be trained, the model training unit 102 calculates a loss function L defined by the error between the predicted value Y^ of the model based on the input data [X, S] and the correct answer label Y corresponding to the input data [X, S], and performs training by back propagation in a manner that minimizes the loss function L. However, the present disclosure is not limited to a specific learning algorithm. Then, the model training unit 102 stores the model parameters obtained as a result of the training in the model parameter holding unit 103.

[0104] Next, the inference phase will be described. The data input unit 112 inputs the data to be inferred. The input data processing unit 113 performs data processing in such a way that the data input from the data input unit 112 has a data format [X, S] that can be input to the trained model, and inputs the data to the inference unit 111. Then, the inference unit 111 outputs an inference result Y^ for the input data [X, S] by using the model (i.e., the trained model) in which the model parameters read from the model parameter holding unit 103 are set (step S211).

[0105] Next, the model bias determination unit 123 analyzes the inference result performed by the inference unit 111 using the trained model and determines whether the fairness of the trained model is ensured, that is, the model bias. In a second embodiment (described later), XAI is used to calculate the influence function of each sensitive attribute data item for the inference result of the training model (step S212), and it is detected whether there is a data item with a low influence score for the inference result (step S213).

[0106] Then, in the case where there is a data item with a low influence score, that is, in the case where the model deviation determination unit 123 determines that there is a model deviation (yes in step S213), the model deviation mitigation unit 124 performs a process for mitigating the model deviation. In the second embodiment of the present disclosure, the model deviation mitigation unit 124 adds input data including the sensitive attribute value with a low influence score determined by the model deviation determination unit 123 for the inference of the trained model (step S214), (step S215), instructs the model training unit 102 to retrain the model by using the added input data, and causes the model training unit 102 to retrain the model (step S215). As a result, the model is retrained so that even if the influence of the sensitive attribute value with a low influence score increases, the model deviation caused by the sensitive attribute can be mitigated.

[0107] After executing the model bias mitigation process in steps S215 to S216, the process returns to step S211, and inference using the model after model bias mitigation and model bias determination is repeated. Then, if it is determined that there is no model bias (No in step S213), it is determined that a fair model without bias is obtained, and the process ends.

[0108] C. First Implementation

[0109] In this Section C, a technique for calculating the bias potential in a dataset used for model training by using XAI to mitigate data bias is described as a first embodiment of the present disclosure. By applying XAI, it is possible to describe how to achieve data bias mitigation in a human-understandable manner.

[0110] C-1. Bias Mitigation Mechanism

[0111] First, the data deviation mitigation process performed by the data deviation mitigation unit 121 will be described in detail.

[0112] The data bias mitigation unit 121 predicts sensitive attributes in the input data based on the non-sensitive attribute data in the input data by using the trained prediction model. Subsequently, the data bias mitigation unit 121 uses XAI to calculate the basis for determining sensitive attribute data from the non-sensitive attribute data using the prediction model, and modifies the correct answer label corresponding to the input data based on the XAI calculation result, thereby mitigating the bias of the input data caused by potential sensitive attributes in the dataset used for training.

[0113] Figure 3 The functional configuration of the data deviation mitigation unit 121 is schematically shown. The data deviation mitigation unit 121 shown includes a pre-processing unit 301, a prediction unit 302, a contribution calculation unit 303, and a label modification unit 304.

[0114] The preprocessing unit 301 performs preprocessing for performing data deviation mitigation processing on the dataset [Y, X, S] extracted from the dataset holding unit 101. Specifically, the preprocessing unit 301 generates input data X including non-sensitive attribute data by removing sensitive attribute data S from the input data [X, S] of the dataset.

[0115] The prediction unit 302 predicts original sensitive attribute data Ŝ from non-sensitive attribute data X by using a trained prediction model for predicting sensitive attributes.

[0116] The contribution calculation unit 303 calculates the contribution of each data item included in the non-sensitive attribute data X of the input data to the prediction result Ŝ of the prediction model.

[0117] In the first embodiment of the present invention, the contribution calculation unit 303 uses Shapley additive interpretation (SHAP) (see non-patent document 2), which is one of the XAI technologies used for model visualization. SHAP is an algorithm that calculates the contribution of each feature of the target data (each feature value included in the target data) to the prediction result of the AI. By using SHAP, when the prediction model predicts the original sensitive attribute data S based on the non-sensitive attribute data X, the contribution of each data item included in the non-sensitive attribute data X is calculated as a Shapley value based on cooperative game theory.

[0118] The label modification unit 304 modifies the correct answer label Y corresponding to the input data [X, S] according to each contribution of each data item included in the non-sensitive attribute data X, thereby reducing the potential bias caused by the sensitive attribute in the data set [Y, X, S]. Then, the corrected correct answer label Y o The dataset [Y o , X, S] is written back to the dataset holding unit 101. For example, in the case where the sensitive attribute is "gender", the label modification unit 304 changes the correct answer label of any one of male and female in which the value of some data items with higher Shapley values ​​(or data items with the highest Shapley value) among the non-sensitive attributes is equal to or greater than a predetermined threshold, and generates a dataset in which the bias caused by gender is mitigated.

[0119] The expected model training unit 102 is trained by using the dataset [Y 0 , X, S] performs model training without bias caused by sensitive attributes, in the dataset [Y 0 , X, S], the correct answer label Y is corrected by alleviating the bias caused by the potential sensitive attributes in the non-sensitive attribute data by the data bias mitigation unit 121 as described above.

[0120] In addition, by using a GUI to present the Shapley values ​​calculated by the contribution calculation unit 303 for each data item of the non-sensitive attribute data X, the values ​​of each data item of the non-sensitive attribute data X before and after correction by the label modification unit 304, etc., it is possible to visualize how to reduce the deviation of the input data, so that humans can understand it.

[0121] Note that XAI applied to the contribution calculation unit 303 is not limited to SHAP. For example, other methods for calculating the importance of each variable, such as Locally Explainable Model-Agnostic Explanations (LIME), can be applied. LIME is an algorithm that estimates a certain input data item (feature) as having "moderate importance" if the output result of the neural network is reversed or greatly changed when the item is changed (see Non-Patent Document 3).

[0122] C-2. Fairness

[0123] This section, C-2, describes the definition of fairness in AI technology. The absence of bias—that is, the AI's predicted distribution remains unchanged by sensitive attributes—means it is fair. Furthermore, sensitive attribute data S essentially includes gender, race, and age. However, depending on the application, any of gender, race, and age may be omitted, or a portion of non-sensitive attribute data may be considered sensitive attribute data.

[0124] For example, when predicting the results of a company recruitment test, if the distribution of predicted scores varies depending on sensitive attributes such as race or gender, the recidivism rate of former criminals, or loan examination results, the model will have difficulty eliminating social bias, which is unfair. In this field, definitions of fairness such as demographic parity, balanced odds, and equal opportunity are well-known.

[0125] Demographic parity is a definition of fairness where the same prediction distribution should be used regardless of the sensitive attributes. For example, in a company’s human resources recruitment, Figure 4 As shown, fairness is defined as the ratio between employment and rejection being the same regardless of the sensitive attribute "gender" (alternatively, the difference between the ratios is equal to or less than a predetermined threshold), and the predicted label ratio is the criterion of fairness. Assuming that the predicted value obtained by the trained model is Y^ and the sensitive attribute is S, the demographic parity is expressed by the following formula (1).

[0126] [Mathematical formula 1]

[0127]

[0128] Furthermore, the definition of balanced odds is fair, i.e. the ratio between the true positive rate (TPR) and the false positive rate (FPR) is the same regardless of the sensitive attribute (alternatively, the difference between the rates is equal to or less than a predetermined threshold). For example, in the prediction of whether a former offender will reoffend, such as Figure 5 As shown, the ratio between the false positive rate (FPR) for former offenders who did not reoffend and the true positive rate (TPR) for former offenders who were predicted to reoffend and actually reoffend is the same, regardless of the sensitive attribute "race" (alternatively, the difference between the ratios is equal to or less than a predetermined threshold). Assuming that the target (whether a certain event actually occurred) is Y, the predicted value obtained by the trained model is Y^, the sensitive attribute is S, and y belongs to {0, 1}, the probability of equilibrium is expressed by the following formula (2).

[0129] [Mathematical formula 2]

[0130]

[0131] Furthermore, equal chance is the definition of fairness, i.e., the ratio of true positive ratios (TPRs) is the same regardless of the sensitive attribute (alternatively, the difference between the ratios is equal to or less than a predetermined threshold), and is different from equal chance, which does not take into account the ratio of FPRs. For example, in the prediction of whether a debtor of a loan will repay the debt, such as Figure 6As shown, regardless of the sensitive attribute "gender" (alternatively, the difference between the ratios is equal to or less than a predetermined threshold), the same ratio of the true positive rate (TPR) for cases where debt repayment is predicted and actually repaid is a fair standard. Unlike the above equal chance, the ratio of the false positive rate (FPR) for cases where debt repayment is predicted and actually has not been repaid is not considered. Assuming that the target (whether a certain event actually occurred) is Y, the predicted value obtained by the trained model is Y^, the sensitive attribute is S, and y belongs to {0, 1}, the equal chance is expressed by the following formula (3).

[0132] [Mathematical formula 3]

[0133]

[0134] In addition to the above, there are definitions of fairness such as fairness through perception where “similar individuals should receive similar predicted values” and fairness through indefiniteness where “fairness should be achieved even without explicitly using sensitive attributes”.

[0135] In any definition of fairness, according to the first embodiment of the present disclosure, it is desirable to perform model training without bias caused by sensitive attributes, wherein by using the dataset [Y 0 , X, S] satisfies each of the above formulas (1) to (3) in each application, where the correct answer label Y is corrected in a manner that mitigates the bias caused by potential sensitive attributes in non-sensitive attribute data.

[0136] Furthermore, according to the first embodiment of the present disclosure, since the non-sensitive attribute data X is corrected using the XAI technology, it is possible to visualize how the bias of the input data is mitigated by presenting the calculation results of the XAI so that humans can understand.

[0137] C-3. Explainable Data Bias Mitigation Methods

[0138] Figure 7 FIG. 1 shows the transformation of the distribution of the data set [Y, X, S] by bias mitigation (by the data bias mitigation unit 121) in the first embodiment of the present invention. Figure 7 In the figure, the vertical plane represents the fairness subspace 701, and the horizontal plane represents the model distribution subspace 702. The beam where the planes of the fairness subspace 701 and the model distribution subspace 702 intersect each other becomes the model subspace 703 that satisfies the fairness constraint. The random variable S is a sensitive attribute, such as gender or race, the random variable X is a non-sensitive attribute, the random variable Y is the correct answer label, and the random variable Y o The random variable Y is the correct answer label subject to data bias mitigation. o is the correct answer label Y that is subject to data bias mitigation oThe sensitive attribute S is a feature vector having each data item included in the sensitive attribute as an element, and the non-sensitive attribute X is a feature vector having all data items other than the sensitive attribute S in the input data (explanatory variable) as elements.

[0139] Pr[Y, X, S] is the true distribution of a data set including original input data and observations Y. The original input data includes non-sensitive attribute data X and sensitive attribute data. First, based on the data of the true distribution Pr[Y, X, S], a prediction model (the prediction model used by the prediction unit 302) is created with the sensitive attribute S set as the target variable based on the non-sensitive attribute X.

[0140] In the data deviation mitigation unit 121, when the prediction unit 320 predicts the sensitive attribute S from the non-sensitive attribute X by using the prediction model (Pr[S|X]), the relationship between the data items of each attribute is obtained. Then, the contribution calculation unit 303 obtains the Shapley value by using SHAP (see non-patent document 2) as XAI, so that people can understand the explanation of the determination basis of the prediction model. The Shapley value is a numerical value that represents the degree of influence of each factor on the prediction of the collaborative game theory. By calculating the Shapley value of the prediction model, people can understand the contribution of each data item included in the non-sensitive attribute X to the sensitive attribute. For example, in the case where "gender" as a sensitive attribute is set as the target variable of the prediction model, the contribution of each of the data items x1, x2, etc. of the non-sensitive attribute X as an explanatory variable is indicated by the Shapley value, so that it is possible to clearly indicate what is the most contributing explanatory variable.

[0141] It can be considered that the explanatory variables that contribute to the prediction with sensitive attributes as target variables have some influence as data bias. Therefore, the label modification unit 304 can reduce the bias caused by sensitive attributes by modifying the correct answer labels corresponding to the data items with higher Shapley values ​​among the data items x1, x2, etc. included in the non-sensitive attribute data X.

[0142] Reference again Figure 7 , obtain the real and fair data distribution Pr[Y 0 ,X,S] of the fair subspace 701, the true and fair data distribution Pr[Y 0 , X, S] includes the correct answer label Y modified by the label modification unit 304 o The subspace where the model subspace 702 and the fairness subspace 703 intersect each other becomes the fair model subspace 703, which includes the fair data distribution Pr[Y o ^, X, S].

[0143] Here, we will describe an example of learning loan inspection results by using the above method to mitigate data bias in the dataset [Y, X, S]. In this case, the correct answer label has two values: 1 (loan allowed) and 0 (loan not allowed), and to simplify the explanation, the sensitive attribute is only "gender". Figure 17 An example of the Shapley value calculated for each data item of the non-sensitive attribute in the case of predicting “gender” as the sensitive attribute S from the non-sensitive attribute X is shown.

[0144] exist Figure 17 In the example shown in , the non-sensitive attribute X includes five data items: "Credit amount", "Responsible person", "Employment = A72", "Investment as a percentage of income", and "Housing = A151". Then, since the Shapley value of "Credit amount" is the highest, it can be understood that the contribution to gender is the largest. Therefore, in the input data [X], in the case where the gender is male, the label modification unit 304 reverses the value of the correct answer label Y (lending inspection result), S] where the value of "Credit amount" is equal to or greater than a predetermined threshold (for example, an average value), or in the case where the gender is female in such input data, reverses the value of the correct answer label Y (loan inspection result) [X, S] to obtain the modified correct answer label Y o , and generate datasets that mitigate gender bias [Y o , X, S].

[0145] The present inventors have obtained experimental results showing that the statistical parity error can be improved by applying the above method, that is, the data bias between males and females can be reduced. The method according to an embodiment of the present disclosure is characterized in that the explanatory variables that contribute to the bias reduction can be expressed in a manner that is intuitively understandable to humans.

[0146] C-4. Configuration of Information Processing Device

[0147] This Section E describes an information processing device for implementing the training system 100 according to an embodiment of the present disclosure. Figure 8 1 shows a configuration example of the information processing device 2000. The training system 100 can be implemented using one information processing device 2000, or the training system 100 can be implemented using two or more information processing devices 2000. For example, for the training phase and the inference phase, the constituent elements of the training system 100 can be divided into two, and the constituent elements of each phase can be implemented using one information processing device 2000.

[0148] exist Figure 8The information processing device 2000 shown in the figure includes a CPU 2001, a read-only memory (ROM) 2002, a random access memory (RAM) 2003, a host bus 2004, a bridge 2005, an expansion bus 2006, an interface unit 2007, an input unit 2008, an output unit 2009, a storage unit 2010, a drive 2011, and a communication unit 2013.

[0149] The CPU 2001 functions as an operation processing device and a control device, and controls the overall operation of the information processing device 2000 according to various programs. Considering the computational load when the information processing device 2000 operates in the training phase of the training system 100, it is desirable that the CPU 2001 is a multi-core CPU, and that the information processing device 2000 includes a multi-core processor such as a GPU or a GPGPU in addition to the CPU 2001. However, for convenience, these are hereinafter referred to simply as the CPU 2001.

[0150] The ROM 2002 stores programs (such as a basic input / output system) and calculation parameters to be used by the CPU 2001 in a nonvolatile manner. The RAM 2003 is used to load programs used in the execution of the CPU 2001 and temporarily store parameters such as working data that change appropriately during the execution of the program. Examples of programs loaded into the RAM 2003 and executed by the CPU 2001 include various application programs, an operating system (OS), and the like.

[0151] The CPU 2001, the ROM 2002, and the RAM 2003 are interconnected via a host bus 2004 including a CPU bus or the like. The CPU 2001 then operates in conjunction with the ROM 2002 and the RAM 2003 to execute various application programs under the execution environment provided by the OS, thereby enabling various functions and services to be implemented. In the case where the information processing apparatus 2000 is a personal computer, the OS is, for example, Windows or Unix of Microsoft Corporation. In the case where the information processing apparatus 2000 is an information terminal such as a smartphone or a tablet computer, the OS is, for example, iOS of Apple Corporation or Android of Google Corporation. In addition, the application program includes an application for operating as a functional module such as a data deviation mitigation unit 121, a model training unit 102, an inference unit 111, an input data processing unit 113, a model deviation determination unit 123, or a model deviation mitigation unit 124.

[0152] The host bus 2004 is connected to the expansion bus 2006 via the bridge 2005. The expansion bus 2006 is, for example, a Peripheral Component Interconnect (PCI) bus or PCI Express, and the bridge 2005 is based on the PCI standard. However, the information processing apparatus 2000 does not necessarily have a configuration in which circuit components are separated by the host bus 2004, the bridge 2005, and the expansion bus 2006, and thus can be configured in such a way that almost all circuit components are interconnected using a single bus (not shown).

[0153] The interface unit 2007 connects peripheral devices such as the input unit 2008, the output unit 2009, the storage unit 2010, the drive 2011, and the communication unit 2013 according to the standard of the expansion bus 2006. Figure 8 All peripheral devices shown in the figure are not necessarily required, and the information processing device 2000 may further include another peripheral device (not shown). In addition, the peripheral device may be built into the main body of the information processing device 2000, or some peripheral devices may be externally connected to the main body of the information processing device 2000.

[0154] The input unit 2008 includes an input control circuit, etc., which generates an input signal based on the input from the user and outputs the input signal to the CPU 2001. In the case where the information processing device 2000 is a personal computer, the input unit 2008 may include a keyboard, a mouse, and a touch panel, and may also include a camera and a microphone. In addition, in the case where the information processing device 2000 is an information terminal such as a smartphone or a tablet, the input unit 2008 is, for example, a touch panel, a camera, or a microphone, and may further include another mechanical operator such as a button.

[0155] The output unit 2009 includes a sound output device such as a speaker and headphones. In addition, the output unit 2009 also includes, for example, a display device such as a liquid crystal display (LCD) device, an organic electroluminescent (EL) display device, and a light emitting diode (LED).

[0156] The storage unit 2010 stores files and various data such as programs (applications, OS, etc.) to be executed by the CPU 2001. The data stored in the storage unit 2010 may include a corpus of normal speech and whisper (as described above) for training a neural network. Although the storage unit 2010 includes, for example, a large-capacity storage device such as a solid-state drive (SSD) or a hard disk drive (HDD), it may include an external storage device.

[0157] The removable recording medium 2012 is a cartridge-type storage medium such as a micro SD card. The drive 2011 performs read and write operations on the removable recording medium 113 loaded therein. The drive 2011 outputs data read from the removable recording medium 2012 to the RAM 2003 and the storage unit 2010, and writes data on the RAM 2003 and the storage unit 2010 to the removable recording medium 2012.

[0158] The communication unit 2013 is a device that performs wireless communication such as Wi-Fi (registered trademark), Bluetooth (registered trademark) or a cellular communication network such as 4G or 5G. In addition, the communication unit 2013 may include a terminal such as a universal serial bus (USB) or a high-definition multimedia interface (HDMI (registered trademark)), and may further include a function of performing communication with a USB device such as a scanner or printer, a display, etc.

[0159] D. Second Implementation

[0160] In this section D, as a second embodiment of the present disclosure, a technique for visualizing model bias by calculating an influence function (see non-patent document 3) of a data item for each sensitive attribute of an inference result obtained by a trained model will be described. In the second embodiment, a technique for mitigating model bias based on the calculated influence function can be further implemented. That is, the model bias caused by sensitive attributes can be mitigated by adding input data of sensitive attributes determined to have low influence scores and retraining the model. In addition, by applying XAI, it is possible to describe how to achieve data bias mitigation in a human-understandable manner.

[0161] For example, when an AI-enabled image sensor is installed and person detection is performed from images captured by the image sensor using AI, the image sensor further calculates an influence function for each sensitive attribute data item in the detection result. It was found that negative bias was applied to data items with low influence scores in person detection, meaning that model fairness was not ensured. Therefore, AI model bias caused by sensitive attributes can be mitigated by adding images with the same sensitive attributes and retraining the AI.

[0162] D-1. Configuration of imaging device

[0163] Figure 9The functional configuration of an imaging device 900 to which the second embodiment of the present disclosure is applied is schematically illustrated. The illustrated imaging device 900 includes an optical unit 901, a sensor unit 902, a sensor control unit 903, an inference unit 904, a memory 905, a visual recognition processing unit 906, an output control unit 907, and a display unit 908. For example, a complementary metal oxide semiconductor (CMOS) image sensor can be formed by integrating the sensor unit 902, the sensor control unit 903, the inference unit 904, and the memory 905 using a CMOS. However, the imaging device 900 may be an infrared light sensor or other type of light sensor that captures images using infrared light.

[0164] The optical unit 901 includes, for example, a plurality of optical lenses for converging light from a subject onto the light-receiving surface of the sensor unit 902, an aperture mechanism for adjusting the size of the aperture relative to the size of the incident light, and a focusing mechanism for adjusting the focus of the irradiated light on the light-receiving surface. The optical unit 901 may further include a shutter mechanism for adjusting the time that the light-receiving surface is irradiated with light. The aperture mechanism, focusing mechanism, and shutter mechanism included in the optical unit 901 are configured to be controlled by, for example, the sensor control unit 903. It should be noted that the optical unit 901 may be configured integrally with the imaging device 900, or may be configured to be attached to the main body of the imaging device 900 separately from the imaging device 900 and to be detachable from and replaceable with the main body of the imaging device 900.

[0165] The sensor control unit 903 includes a microprocessor, for example, that controls the reading of pixel data from the sensor unit 902 and outputs image data based on each pixel signal read from each pixel. The pixel data output from the sensor control unit 903 is passed to the inference unit 904 and the visual recognition processing unit 906. In addition, the sensor control unit 903 generates an imaging control signal for controlling the sensor unit 902 and supplies the imaging control signal to the sensor unit 902. The imaging control signal includes information indicating the exposure and analog gain when imaging in the sensor unit 902. The imaging control signal further includes a control signal for executing the imaging operation of the sensor unit 902, such as a vertical synchronization signal or a horizontal synchronization signal.

[0166] The inference unit 904 corresponds to Figure 19. The inference unit 904 is shown as “inference unit 111”. The model parameters supplied from the model parameter holding unit 103 are stored in the memory 905, and the inference unit 904 uses a trained model in which the model parameters read from the memory 905 are set to perform inference using image data after visual recognition processing performed by the visual recognition processing unit 906. However, the inference unit 904 may perform inference on pixel data delivered from the sensor control unit 903. The recognition result of the inference unit 904 is delivered to the output control unit 907.

[0167] The inference performed by the inference unit 904 includes recognition processing of objects contained in the image based on pixel data (such as human detection, facial recognition, image classification, etc.). In addition, the inference unit 904 may further include various predictions, such as determining whether to hire or reject personnel, determining recidivism rates, and load checks.

[0168] In addition, in this embodiment, the inference unit 904 further incorporates XAI and performs Figure 1 The processing corresponding to the "model deviation determination unit 123" in the training model, that is, the processing of analyzing the inference result performed by the training model and determining the model deviation. Specifically, the inference unit 904 calculates the influence function of the sensitive attribute of the person detection result of the training model by using XAI. For example, the inference unit 904 calculates the influence function of the person detection for each attribute value of "male" and "female" for the data item "gender" of the sensitive attribute, and calculates the influence function of the person detection for each attribute value of "white race", "black race", etc. for the data item "race" of the sensitive attribute. Then, the inference unit 904 adds the influence function calculation result to the person detection result, and passes the result to the output control unit 907.

[0169] The visual recognition processing unit 906 performs processing for obtaining an image suitable for human visual recognition on the pixel data transmitted from the sensor control unit 903, and outputs image data including, for example, a set of pixel data. For example, when a color filter is set for each pixel included in the sensor unit 902 and each pixel data has any color information of red (R), green (G), or blue (B), the visual recognition processing unit 906 performs demosaicing processing, white balance processing, etc. In addition, the visual recognition processing unit 906 can instruct the sensor control unit 903 to read the pixel data required for the visual recognition processing from the sensor unit 902. The visual recognition processing unit 906 transmits the image data on which the pixel data has been processed to the output control unit 907. For example, the above-mentioned functions of the visual recognition processing unit 906 are realized by an image signal processor that executes a program pre-stored in a local memory (not shown).

[0170] The output control unit 907 includes, for example, a microprocessor. The output control unit 907 receives image data as a result of the visual recognition processing from the visual recognition processing unit 906, and performs processing for displaying the image data on the display unit 908 or outputting the image data to the outside of the imaging device 900. A user can visually recognize a captured image on the screen of the display unit 908. The display unit 908 may be built into the imaging device 900, like a viewfinder on the back of a digital camera body, or may be externally connected to the imaging device 900 via an HDMI (registered trademark) interface or the like.

[0171] In this embodiment, the output control unit 907 outputs the inference result (human detection result) of the inference unit 904 and the determination basis of the inference (influence function of the sensitive attribute) together with the image data. The display unit 908 can present the inference result and the determination basis together with the captured image. In addition, the output control unit 907 can upload the determination basis for inference (influence function of the sensitive attribute) to the model training server ( Figure 9 not shown).

[0172] D-2. Image Sensor Configuration

[0173] Figure 10 FIG. 1 shows an example of hardware implementation of an image sensor 1000 that integrates a sensor unit 902, a sensor control unit 903, an inference unit 904, and a memory 905. Figure 10 In the example shown in , the sensor unit 902, the sensor control unit 903, the inference unit 904, the memory 905, the visual recognition processing unit 906, and the output control unit 907 are mounted on one chip 1000. However, in Figure 10 In order to avoid confusion in the drawings, the memory 905 and the output control unit 907 are omitted. Figure 16 In the illustrated configuration example, the inference result and its determination basis (the influence function of the sensitive attribute) of the inference unit 904 are output to the outside of the chip 1000 via the output control unit 907. In addition, the inference unit 904 can obtain pixel data or image data to be used for recognition from the sensor control unit 903 via an interface within the chip 1000.

[0174] Figure 11 FIG. 1 shows a hardware implementation example of another image sensor 1100 in which a sensor unit 902, a sensor control unit 903, an inference unit 904, and a memory 905 are integrated. Figure 11In the example shown in , the sensor unit 902, the sensor control unit 903, the visual recognition processing unit 906, and the output control unit 907 are mounted on one chip 1100, and the inference unit 904 and the memory 905 are arranged outside the chip 1100. However, also in Figure 11 In the figure, the memory 1505 and the output control unit 907 are omitted to avoid confusion in the drawing. Figure 11 In the configuration example shown in , the inference unit 904 obtains pixel data or image data to be used for recognition from the output control unit 907 via the communication interface between chips. In addition, the inference unit 904 directly outputs the inference result and its determination basis (the influence function of the sensitive attribute) to the outside. Of course, the following configuration can be adopted: the inference result of the inference unit 904 and its determination basis are returned to the output control unit 907 in the chip 1100 via the communication interface between chips, and are output from the output control unit 907 to the outside of the chip 1100.

[0175] exist Figure 10 In the configuration example shown in , since both the inference unit 904 and the sensor control unit 903 are mounted on the same chip 1000, communication between the inference unit 904 and the sensor control unit 903 can be performed at high speed via the interface in the chip 1000. However, in Figure 11 In the configuration example shown in FIG, since the inference unit 904 is arranged outside the chip 1100, it is easy to replace the inference unit 904, and the trained model can be exchanged by replacement. However, communication between the inference unit 904 and the sensor control unit 903 needs to be performed via the interface between the chips, which reduces the speed.

[0176] Figure 12 An example is shown in which the image sensor 1000 (or 1100) is formed as a semiconductor chip 1200 having a two-layer structure in which two layers are stacked. An example is shown in which the stacked image sensor 1200 is formed to have a double-layer structure in which semiconductor chips are stacked in two layers. Figure 12 In the structure shown, the pixel unit 1211 is formed in the semiconductor chip 1201 of the first layer, and the memory and logic unit 1212 is formed in the semiconductor chip 1202 of the second layer. Figure 12 As shown on the right side of FIG, a single solid-state imaging element is configured by bonding the first-layer semiconductor chip 1201 and the second-layer semiconductor chip 1202 in such a manner that the first-layer semiconductor chip 1201 and the second-layer semiconductor chip 1202 are in electrical contact with each other.

[0177] The pixel unit 1211 includes at least the pixel array in the sensor unit 902. In addition, the memory and logic unit 1212 includes, for example, the sensor control unit 903, the inference unit 904, the memory 905, the visual recognition processing unit 906, the output control unit 907, and an interface for performing communication with the outside. The memory and logic unit 1212 also includes part or all of the driving circuit that drives the pixel array in the sensor unit 902. In addition, although Figure 12 Not shown, the memory and logic unit 1212 may also include, for example, memory used by the visual recognition processing unit 906 for processing image data.

[0178] Figure 13 1 shows an example in which the image sensor 1000 (or 1100) is formed as a semiconductor chip 1300 having a three-layer structure in which three layers are stacked. Figure 13 In the structure shown, a pixel unit 1311 is formed in the semiconductor chip 1301 of the first layer, a memory unit 1312 is formed in the semiconductor chip 1302 of the second layer, and a logic unit 1313 is formed in the semiconductor chip 1303 of the third layer. Figure 13 As shown on the right side of , a single solid-state imaging element is configured by joining the first-layer semiconductor chip 1301, the second-layer semiconductor chip 1302 and the third-layer semiconductor chip 1303 in such a manner that the first-layer semiconductor chip 1301, the second-layer semiconductor chip 1302 and the third-layer semiconductor chip 1303 are electrically contacted with each other.

[0179] The pixel unit 1311 includes at least the pixel array in the sensor unit 1502. Furthermore, the logic unit 1313 includes, for example, the sensor control unit 903, the inference unit 904, the visual recognition processing unit 906, the output control unit 907, and an interface for communicating with the outside world. The logic unit 1313 further includes part or all of the driver circuitry that drives the pixel array in the sensor unit 902. Furthermore, in addition to the memory 905, the memory unit 1312 may also include, for example, memory used by the visual recognition processing unit 906 to process image data.

[0180] The human detection result and the influence function of the sensitive attribute of the inference unit 904 in the logic unit 1212 (or 1312 ) and the image captured by the pixel unit 1211 (or 1311 ) are simultaneously output from the semiconductor chip 1200 (or 1300 ).

[0181] Figure 16 An example of operation in a digital camera 1600 in which an image sensor 1601 including the multilayered semiconductor chip 1200 (or 1300 ) including a logic unit is mounted is shown.

[0182] When the digital camera 1600 images persons #1 to #3 of different races, genders, ages, etc., the image sensor 1601 simultaneously outputs a captured image of each human body, a human body detection result of the captured image, and an influence function of a data item of each sensitive attribute during human body detection.

[0183] For example, when a person is detected from the first captured image 1611, the image sensor 1601 calculates the influence function of the data item "race: black", "gender: male", or "age: ○" for each sensitive attribute, and outputs the influence function simultaneously with the captured image. Similarly, when a person is detected from the second captured image 1612, the image sensor 1601 calculates the influence function of the data item "race: white", "gender: male", or "age: △○" for each sensitive attribute, and when a person is detected from the third captured image 1613, the image sensor 1601 calculates the influence function of the data item "race: white", "gender: female", or "age: △□" for each sensitive attribute, and outputs the influence function simultaneously with the captured image.

[0184] For example, if the influence score of the sensitive attribute data item "race: black" is significantly low (or, alternatively, if the influence score is less than a predetermined threshold), it can be determined that the trained model for human detection is negatively biased with respect to "black race." In this case, the model bias can be eliminated by adding images of black people and retraining the model to increase the influence score of "black race."

[0185] D-3. System Configuration and Operation

[0186] Figure 14 An operation example of the training system 100 in the case of applying the second embodiment of the present disclosure is shown. In the shown example, the training system 100 is configured as a client-server system including a server on the cloud and an unlimited number of edge devices (clients) receiving services from the server.

[0187] In this server, functional modules related to the model training process are arranged, including a data deviation mitigation unit 121 that mitigates the potential for deviation in a data set and a model training unit 102 that trains a model by using the data set with the deviation mitigated. Then, on the server side, the model parameters obtained as a result of the model training process performed by the model training unit 102 are saved in a model parameter holding unit 103. On the other hand, there are a large number of edge devices on the cloud side. The edge devices are digital cameras, multi-function information terminals such as smartphones or tablet computers, personal computers, etc. Each edge device includes an inference unit 111, and there is also an edge device that includes a model deviation determination unit 123.

[0188] The server provides each edge device with model parameters obtained by training the model using a dataset with reduced bias. The form in which the model parameters are provided is variable. For example, the model parameters can be pre-installed when the edge device product is shipped. In addition, even after the product is shipped, the edge device can download the model parameters from the server at an appropriate timing. In addition, even after the start of use, the model parameters updated on the server side can be appropriately downloaded to the edge device as needed. The update of the model parameters on the server side includes updates accompanied by model bias reduction, and of course, the update of the model parameters on the server side can also include updates for other purposes.

[0189] In each edge device, the model parameters provided by the server are set in the inference unit 111, and prediction processing using the trained model is performed. The task performed by the trained model is, for example, person detection from a captured image from a camera. However, the task can also be facial detection or facial recognition, and can further include determining the likelihood of an employee being hired, the likelihood of a loan inspection, a repeat offender, etc. based on a facial image.

[0190] The edge device including the model deviation determination unit 123 uses the trained model to analyze the inference result performed by the inference unit 111, and determines whether the fairness of the model is ensured, that is, the model deviation. Specifically, the model deviation determination unit 123 uses XAI to calculate the influence function of the sensitive attribute of the inference result of the training model. That is, the model deviation determination unit 123 calculates the influence function of the data item of each sensitive attribute of the inference result of the training model by using XAI, and detects whether there is a data item with a low influence score for the inference result. Then, the model deviation determination unit 123 determines to apply a negative bias to the data item of the sensitive attribute with a low influence score, that is, the fairness of the model cannot be ensured, and uploads the determination result (or, the influence score of the data item of each sensitive attribute) to the server (or, the server can be notified only when it is determined that a model deviation exists).

[0191] The model bias mitigation unit 124 on the server side performs processing to mitigate model bias based on the model bias determination result. The model bias mitigation unit 124 can execute model bias mitigation algorithms such as transfer learning, fine-tuning, and incremental learning, and the model training unit 102 can mitigate the bias of the trained model. Specifically, the model bias mitigation unit 124 enhances the dataset corresponding to the sensitive attributes that the model bias determination unit 123 has determined to have low influence scores for inference by the trained model, and instructs the model training unit 102 to retrain the model using the enhanced dataset. In the case of image data, the model bias mitigation unit 124 can generate an image for enhancement using, for example, computer graphics (CG), or can generate an image using a generative adversary network (GAN). As a result of retraining using the enhanced dataset, the model is updated so that even data items with low influence scores for sensitive attributes are increased in influence, and the model bias caused by the sensitive attributes is mitigated. The model parameters whose model bias has been mitigated through retraining are then accumulated in the model parameter storage unit 103 and provided to each edge device.

[0192] Next, the system operation will be described. Figure 15 Schematically shows the Figure 14 An example of a communication sequence between an edge device and a server in a client server system is shown in FIG. Figure 15 In order to simplify the drawings, edge devices and servers are shown in a one-to-one relationship, but in practice, one server implements a similar communication sequence with a large number of edge devices.

[0193] However, in the following description, it is assumed that a digital camera (alternatively, an information terminal having a digital camera function) is used as an edge device, and an image sensor chip (see FIG. 1 ) having the functions of each of the inference unit 111 and the model deviation determination unit 123 is used. Figure 10 、 Figure 12 and Figure 13 ) is installed on a digital camera. In addition, it is assumed that model parameters trained to detect people from captured images are provided from a server to the edge device.

[0194] First, the server provides the model parameters obtained through learning to the edge device (TR1501). Alternatively, the edge device obtains the model parameters from the server. On the other hand, on the edge device side, the model parameters provided from the server are set in the model used by the inference unit 111 (TR1502), and human detection using the trained model becomes possible.

[0195] Thereafter, when performing an imaging operation in the edge device (TR1503) as a normal digital camera, the inference unit 111 in the image sensor performs a human detection process (TR1504) by using the trained model, and the model bias determination unit 123 performs model bias determination of the human detection result (TR1505) by using XAI. As the model bias determination, the model bias determination unit 123 calculates an influence function of a data item for each sensitive attribute of the human detection result of the trained model by using XAI, and detects whether there is a data item with a low influence score for the human detection result.

[0196] The edge device then notifies (uploads) the model deviation determination results to the server (TR1506). At this point, the edge device may notify the server only of data items with low influence scores (i.e., data items with negative deviation). Alternatively, the edge device may notify the server of all influence functions calculated for each sensitive attribute data item and determine the data items with negative deviation on the server side.

[0197] Upon receiving a notification of the model bias determination result from the edge device, model bias mitigation for training the model is performed on the server side. First, the server augments the dataset corresponding to data items of sensitive attributes with low influence scores in human detection (TR1507). In the case of image data, for example, CG can be used to generate images for enhancement, or GAN can be used to generate images. The server then performs model retraining using the augmented dataset (TR1508), thereby mitigating model bias caused by sensitive attributes.

[0198] The server then provides the model parameters to the edge device after mitigating the model bias (TR1509). For example, the new model parameters can be distributed from the server to the edge device in the form of a product software update.

[0199] After releasing the product, it is difficult to verify and modify the model. Figure 15 The communication sequence shown in the figure allows XAI to be used to verify the bias of the training model used in each product (i.e., edge device) during actual operation after release. Furthermore, retraining can be performed on the server side based on the information about the detected model bias, and the model parameters after mitigating the model bias can be updated for each product. Therefore, in actual operation, if issues with the fairness of the training model become apparent, model bias can be mitigated.

[0200] For example, as referenced in Section D-2 above Figure 16In the described scenario, if the edge device observes a low influence score for the sensitive attribute "race: black" in a training model for human detection, it can determine to apply a negative bias to "black" in the training model for human detection. The edge device notifies the server of this determination. Then, in response to the notification from the edge device, the server can remove the model bias by adding images of black people and retraining the model. Furthermore, the server can provide the edge device with model parameters with an increased influence score for "black."

[0201] Industrial Applicability

[0202] The present disclosure has been described in detail with reference to specific embodiments. However, the present disclosure should not be construed as being limited to the above-described embodiments, and it is apparent that those skilled in the art may modify and replace the embodiments without departing from the gist of the present disclosure. In addition, the effects described herein are merely examples, so that the effects brought about by the embodiments of the present disclosure are not limited and may include additional effects not described herein.

[0203] The present disclosure can be applied to training models that perform processing such as facial detection, facial recognition, person / object detection, and pose estimation, as well as various types of detection, determination, estimation, and recognition, by receiving images or sounds. According to the present disclosure, XAI can be used to calculate potential data bias in a dataset used for model training, and data bias can be mitigated based on the calculation results. Furthermore, according to the present disclosure, model bias can be mitigated by using XAI to calculate model bias, augmenting the negative bias dataset, and retraining the model.

[0204] In short, the present disclosure has been described in an illustrative manner, and the contents described herein should not be interpreted in a restrictive manner. In order to determine the gist of the present disclosure, the claims should be considered.

[0205] The series of processes described in this specification can be executed by hardware, software, or a combination of hardware and software. In the case of executing the processes by software, a program recording the process sequence related to the implementation of the present disclosure is installed and executed in a memory in dedicated hardware incorporated in a computer. Alternatively, the program can be installed in a general-purpose computer capable of executing various types of processes, and the computer can be caused to execute the processes related to the embodiments of the present disclosure.

[0206] The program may be pre-stored in a computer recording medium such as an HDD, SSD, or ROM. Alternatively, the program may be temporarily or permanently stored in a removable recording medium such as a floppy disk, a compact disk read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disk (DVD), a Blu-ray Disc (BD) (registered trademark), a magnetic disk, or a universal serial bus (USB) memory. The use of such a removable recording medium enables the program related to the implementation of the present disclosure to be provided as so-called package software.

[0207] In addition, the program can be transferred from the download site to the computer via a network (such as a wide area network (WAN) represented by a cellular network, a local area network (LAN), or the Internet) in a wireless or wired manner. The computer can receive the transferred program and install it in a large-capacity storage device such as an HDD or SSD in the computer.

[0208] It should be noted that the present disclosure may also have the following configurations.

[0209] (1) An information processing device comprising:

[0210] an acquisition unit that acquires information about bias caused by sensitive attributes included in data or models by using explainable artificial intelligence (XAI); and

[0211] A processing unit performs processing to reduce bias of the data or model based on the acquired information.

[0212] (2) The information processing device according to (1), wherein

[0213] The acquisition unit includes: a preprocessing unit, which removes data items related to sensitive attributes from input data including multiple data items; a prediction unit, which predicts sensitive attributes based on data items of non-sensitive attributes in the input data; and a contribution calculation unit, which calculates the contribution of each data item of non-sensitive attributes in the input data to the prediction result of the prediction unit, and

[0214] The processing unit includes: a data set generating unit, which modifies the correct answer label corresponding to the input data according to the contribution calculation result, and generates a data set that reduces the bias caused by sensitive attributes.

[0215] (3) The information processing device according to (2), wherein

[0216] The processing unit modifies a correct answer label for each sensitive attribute value of input data, the input data including data items with higher contribution in non-sensitive attributes, and generates a data set that mitigates bias caused by the sensitive attributes.

[0217] (4) The information processing device according to (2), wherein

[0218] The processing unit modifies a correct answer label for each sensitive attribute value of input data, the input data including data items having a higher contribution degree equal to or higher than a predetermined threshold in non-sensitive attributes, and generates a data set that mitigates bias caused by the sensitive attributes.

[0219] (5) The information processing device according to (2), wherein

[0220] Sensitive attributes include gender, and

[0221] The processing unit changes the correct answer label for either male or female whose value of the data item with a higher contribution in the non-sensitive attribute is equal to or higher than a predetermined threshold, and generates a data set that reduces bias caused by gender.

[0222] (6) The information processing device according to any one of (2) to (5), wherein

[0223] The prediction unit predicts the sensitive attribute based on the data items of the non-sensitive attribute in the input data by using a prediction model with the sensitive attribute as the target variable.

[0224] (7) The information processing device according to (6), wherein

[0225] The contribution calculation unit calculates the contribution of each data item of the non-sensitive attribute based on a determination basis of a prediction model obtained by using XAI.

[0226] (8) The information processing device according to any one of (6) and (7), wherein

[0227] The contribution calculation unit calculates the contribution based on the Shapley value of the data item of each non-sensitive attribute by using Shapley Additive Interpretation (SHAP).

[0228] (9) The information processing device according to any one of (2) to (8), further comprising:

[0229] A training unit that trains a model by using the dataset generated by the dataset generating unit and having reduced bias; and

[0230] The inference unit performs inference by using the model trained by the training unit.

[0231] (10) The information processing device according to any one of (1) to (9), wherein

[0232] The acquisition unit further includes an influence function calculation unit that calculates an influence function of the sensitive attribute based on a result of determining the input data through the trained model.

[0233] (11) The information processing device according to (10), wherein

[0234] The input data is the image captured by the image sensor,

[0235] The trained model is a model trained to detect people from images, and

[0236] The influence function calculation unit calculates an influence function of a sensitive attribute when a person is detected from an image by the trained model.

[0237] (12) The information processing device according to any one of (10) and (11), wherein

[0238] The processing unit also performs model retraining by adding images that affect sensitive attributes with low function values.

[0239] (13) An information processing method comprising:

[0240] By using explainable artificial intelligence (XAI) to obtain information about biases caused by sensitive attributes included in data or models; and

[0241] Based on the acquired information, a process of mitigating bias in the data or model is performed.

[0242] (14) A computer program described in a computer-readable format, which causes a computer to:

[0243] an acquisition unit that acquires information about bias caused by sensitive attributes included in data or models by using explainable artificial intelligence (XAI); and

[0244] A processing unit performs processing to reduce bias of the data or model based on the acquired information.

[0245] (15) An image sensor comprising:

[0246] Multilayer semiconductor chips,

[0247] wherein a sensor unit including a pixel array and capturing an image is mounted on a first layer, and a memory unit and a logic unit are mounted on a second layer and subsequent layers, the memory unit stores the captured image of the sensor unit, and the logic unit controls driving of the sensor unit and processes the captured image of the sensor unit, and

[0248] The logic unit includes an inference unit that performs inference on a captured image by using a trained model, and a determination basis calculation unit that calculates a determination basis for a result of the inference performed by the trained model.

[0249] (16) The image sensor according to (15), wherein

[0250] The inference unit detects people from the captured image by using the trained model, and

[0251] In the case where a person is detected from an image by the trained model, a basic computing unit is determined to calculate an influence function of a sensitive attribute.

[0252] Reference Symbol List

[0253] 100 Training System

[0254] 101 Data set holding unit

[0255] 102 Model Training Unit

[0256] 103 Model parameter holding unit

[0257] 111 Inference Unit

[0258] 112 Data Input Unit

[0259] 113 Input Data Processing Unit

[0260] 121 Data Bias Mitigation Unit

[0261] 122 Data Supplement Unit

[0262] 123 Model deviation determination unit

[0263] 124 Model Bias Mitigation Unit

[0264] 130 Dataset Collection Unit

[0265] 301 Preprocessing Unit

[0266] 302 prediction unit

[0267] 303 Contribution Calculation Unit

[0268] 304 Tag Modification Unit

[0269] 900 Imaging Device

[0270] 901 Optical Unit

[0271] 902 sensor unit

[0272] 903 Sensor Control Unit

[0273] 904 Inference Unit

[0274] 905 Memory

[0275] 906 Visual Recognition Processing Unit

[0276] 907 Output Control Unit

[0277] 908 display unit

[0278] 1600 digital camera

[0279] 1601 Image Sensor

[0280] 2000 Information Processing Device

[0281] 2001CPU

[0282] 2002ROM

[0283] 2003RAM

[0284] 2004 Host Bus

[0285] 2005 Bridge

[0286] 2006 Expansion Bus

[0287] 2007 Interface Unit

[0288] 2008 Input Unit

[0289] 2009 Output Unit

[0290] 2010 Storage Unit

[0291] 2011 Driver

[0292] 2012 Removable Recording Media

[0293] 2013 Communications Unit.

Claims

1. An information processing device, comprising: an acquisition unit that obtains information about biases caused by sensitive attributes included in data or models by using explainable artificial intelligence (XAI); as well as A processing unit performs processing to reduce bias of the data or the model based on the acquired information.

2. The information processing device according to claim 1, wherein The acquisition unit includes: a pre-processing unit, which removes data items related to the sensitive attribute from input data including multiple data items; a prediction unit, which predicts the sensitive attribute based on data items of non-sensitive attributes in the input data; and a contribution calculation unit, which calculates the contribution of each data item of the non-sensitive attribute in the input data to the prediction result of the prediction unit, and The processing unit includes: a data set generating unit, which modifies the correct answer label corresponding to the input data according to the contribution calculation result, and generates a data set that reduces the deviation caused by the sensitive attribute.

3. The information processing device according to claim 2, wherein: The processing unit modifies the correct answer label for each sensitive attribute value of the input data, the input data including the data items having higher contributions among the non-sensitive attributes, and generates a data set that mitigates the bias caused by the sensitive attributes.

4. The information processing device according to claim 2, wherein: The processing unit modifies the correct answer label for each sensitive attribute value of the input data, the input data including the data items having a higher contribution degree of the non-sensitive attributes being equal to or higher than a predetermined threshold, and generates a data set that mitigates the bias caused by the sensitive attributes.

5. The information processing apparatus according to claim 2, wherein: The sensitive attributes include gender, and The processing unit changes the correct answer label for either male or female whose value of the data item with a higher contribution in the non-sensitive attribute is equal to or higher than a predetermined threshold, and generates a data set that reduces bias caused by gender. The information processing apparatus according to claim 2 , wherein: The prediction unit predicts the sensitive attribute from the data item of the non-sensitive attribute in the input data by using a prediction model having the sensitive attribute as a target variable.

7. The information processing apparatus according to claim 6, wherein: The contribution calculation unit calculates the contribution of each of the data items of the non-sensitive attribute based on a determination basis of the prediction model obtained by using the XAI.

8. The information processing apparatus according to claim 6, wherein: The contribution calculation unit calculates the contribution based on the Shapley value of each data item of the non-sensitive attribute by using Shapley Additive Interpretation (SHAP).

9. The information processing apparatus according to claim 2, further comprising: a training unit that trains the model by using the dataset generated by the dataset generating unit and having reduced bias; as well as An inference unit that performs inference by using the model trained by the training unit.

10. The information processing apparatus according to claim 1, wherein: The acquisition unit further includes an influence function calculation unit configured to calculate an influence function of the sensitive attribute based on a result of determining input data through a trained model. The information processing apparatus according to claim 10 , wherein: The input data is an image captured by an image sensor, The trained model is a model trained to detect a person from the image, and the influence function calculation unit calculates the influence function of the sensitive attribute when a person is detected from the image by the trained model.

12. The information processing apparatus according to claim 10, wherein: The processing unit also performs model retraining by adding images of the sensitive attribute having low impact function values.

13. An information processing method, comprising: By using explainable artificial intelligence (XAI) to obtain information about biases caused by sensitive attributes included in data or models; as well as Processing for mitigating bias in the data or the model is performed based on the acquired information.

14. A computer program described in a computer readable format, the computer program causing a computer to function as: an acquisition unit that acquires information about bias caused by sensitive attributes included in data or models by using explainable artificial intelligence (XAI); and A processing unit performs processing to reduce bias of the data or the model based on the acquired information.

15. An image sensor comprising: Multilayer semiconductor chips, In which, a sensor unit including a pixel array and capturing an image is installed on a first layer, and a memory unit and a logic unit are installed on a second layer and subsequent layers, the memory unit stores the captured image of the sensor unit, the logic unit controls the driving of the sensor unit and processes the captured image of the sensor unit, and the logic unit includes an inference unit and a determination basis calculation unit, the inference unit performs inference on the captured image by using a trained model, and the determination basis calculation unit calculates a determination basis for a result of the inference performed by the trained model.

16. The image sensor according to claim 15, wherein: The inference unit detects a person from a captured image by using the trained model, and In a case where a person is detected from the image by the trained model, the determination basis calculation unit calculates an influence function of a sensitive attribute.

Images