Fair Deep Reinforcement Learning via Bias Weight Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep reinforcement learning agents often exhibit bias in their actions due to biased training data, leading to unfair and unethical behavior, especially when performing similar tasks in different environments.
Innovation Solution
A computer-implemented method that utilizes semi-supervised training to determine bias weights and merges them with non-bias weights in an artificial neural network, employing a limit function to control the merging process and reduce bias, thereby generating a fair deep reinforcement learning model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deep reinforcement learning agents are trained using standard supervised learning methods, then they can learn to perform tasks efficiently, but they exhibit bias in their actions due to biased training data
Solution Approach 1:
The neural network is divided into two separate processing paths: a biased path that processes features potentially containing bias, and a non-biased path that processes features specifically selected to be free from bias. This segmentation allows the system to handle biased training data while explicitly modeling and removing bias through the parallel non-biased path.
Solution Approach 2:
The patent extracts and isolates the bias component from the training data by identifying protected attributes (such as race, gender, age) and removing their influence during the feature processing stage. The non-biased path explicitly excludes features correlated with protected attributes, thereby extracting the harmful bias element from the learning process.
2Adaptability or versatility
If agents are trained to maximize reward in different environments, then they adapt to various conditions, but bias persists across different environments
Solution Approach 1:
The patent implements a universal fairness mechanism that operates across multiple environments and tasks. The non-biased path and bias removal mechanism are designed to be environment-agnostic, applying the same bias mitigation principles regardless of the specific environment or task, thereby ensuring consistent fair behavior across diverse conditions.
Solution Approach 2:
The patent applies different processing qualities to different feature paths: the biased path processes all available features for maximum adaptability, while the non-biased path applies a filtered quality that excludes biased features. This local quality differentiation allows the system to maintain environmental adaptability through the biased path while ensuring fairness through the non-biased path.
Data Source
AI summary
Fair deep reinforcement learning is provided. A microstate of an environment and reaction of items in a plurality of microstates within the environment are observed after an agent performs an action in the environment. Semi-supervised training is utilized to determine bias weights corresponding to the action for the microstate of the environment and the reaction of the items in the plurality of microstates within the environment. The bias weights from the semi-supervised training are merged with non-bias weights using an artificial neural network. Over time, it is determined where bias is occurring in the semi-supervised training based on merging the bias weights with the non-bias weights in the artificial neural network. A deep reinforcement learning model that decreases reliance on the bias weights is generated based on determined bias to increase fairness.