Bias Evaluation Program for Multi-Group Fairness Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Typical bias evaluation methods face reliability issues when evaluating fairness between multiple groups, as they often include biases related to unprivileged attributes, leading to inaccurate discrimination detection.
Innovation Solution
The proposed bias evaluation program and device classify data into groups based on multiple attributes, compare positive example rates, and execute bias evaluation by excluding the influence of unprivileged attributes, ensuring the validity and reliability of the bias evaluation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If typical AI bias evaluation devices compare privileged group and unprivileged group using binarization, then bias evaluation can be executed quickly and simply, but the evaluation accuracy deteriorates when fairness issues exist between multiple groups
Solution Approach 1:
The patent segments the evaluation process into two distinct phases: (1) initial binarization evaluation for quick screening, and (2) multi-group detailed evaluation for accurate detection. This segmentation allows the system to quickly identify potential bias cases while maintaining high detection accuracy through subsequent detailed analysis of multiple groups.
Solution Approach 2:
The patent implements a dynamic evaluation approach where the evaluation method adapts based on the detection results. When binarization evaluation detects a bias, the system dynamically transitions to multi-group evaluation to verify and analyze the specific nature of the bias, making the evaluation process flexible and context-dependent.
2Measurement precision
If bias evaluation is performed by comparing target group with non-target group, then fairness issues between multiple groups can be detected, but the evaluation process becomes complex and computationally intensive
Solution Approach 1:
The patent performs preliminary binarization evaluation before conducting detailed multi-group analysis. This preliminary action filters out cases that do not require complex multi-group evaluation, reducing the overall computational burden and process complexity while maintaining the capability to detect multi-group fairness issues when necessary.
Solution Approach 2:
The patent applies partial multi-group evaluation only to cases where binarization evaluation detects potential bias, rather than performing complete multi-group evaluation on all cases. This partial action approach maintains high detection capability for multi-group fairness issues while avoiding unnecessary computational complexity in cases where simple binarization suffices.
3Adaptability or versatility
If unprivileged attributes are included in bias evaluation, then comprehensive fairness assessment can be performed, but false detection of privileged attribute biases occurs
Solution Approach 1:
The patent segments the attribute analysis into privileged attributes (for final bias determination) and unprivileged attributes (for contextual understanding only). By separating the evaluation of these attribute types and using unprivileged attributes only for reference in multi-group analysis rather than as direct bias indicators, the system maintains comprehensive assessment capability while preventing false detections.
Solution Approach 2:
The patent uses unprivileged attributes as intermediary elements in the evaluation process. These attributes serve as mediators to understand the context and relationships between groups without being directly used as basis for bias determination. This intermediary role allows comprehensive fairness assessment while preventing false detection of privileged attribute biases.
Data Source
AI summary
A recording medium storing a program for causing a computer to execute: classifying a plurality of pieces of data into a plurality of groups, based on a first attribute of multiple attributes included in each of the plurality of pieces of data; comparing a positive example rate of data included in a first group of the plurality of groups with a positive example rate of specific data in which a value of a second attribute of the multiple attributes in data included in a second group of the plurality of groups is included in a range according to a distribution of the value of the second attribute of the data included in the first group; and executing bias evaluation on data based on comparison between other groups other than the second group of the plurality of groups and the first group, based on a result of the comparing processing.