Language Model Bias Estimation for Reliable Text Generation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing language models, such as GPT-3, generate biased outputs due to learning from hate speech and politically/ethically biased information, posing reliability issues and hindering commercialization.

Innovation Solution

A deep learning-based text generation model that incorporates a bias estimation model and a bias determination unit to predict and remove biased tokens by using a vocabulary and semantic unit bias estimation, ensuring the generated text does not exceed a preset critical bias value.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If a high-capacity language model learns from huge corpora to improve text generation capability, then the text generation performance is improved, but biased information and hate speeches are learned and generated

Engineering Contradiction:

Improvetext generation capabilityVSAvoidbias-free generation

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent introduces a bias estimation model as an intermediary component between the text generation model and the output. This mediator estimates the bias level of generated text and enables bias removal through human intervention, allowing the system to maintain high generation capability while improving reliability by filtering out biased content.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If all data used for learning is reviewed to remove bias, then biased information is reduced, but the process is not efficient and time-consuming

Engineering Contradiction:

Improvebias reductionVSAvoiddata review time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent implements preliminary bias estimation by training a bias estimation model on labeled data before actual text generation. This preliminary action enables the system to quickly estimate bias in generated text without requiring manual review of all training data, thus reducing time loss while maintaining reliability.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system performs self-service bias detection through the automated bias estimation model that evaluates generated text for bias without requiring external manual review of training data. The model serves itself by identifying and flagging biased content automatically.

Inventive Principle:

Principle #25Self-service

3Reliability

If post-processing is applied to review and feedback bias in generated results, then biased results are reduced, but additional processing time is required

Engineering Contradiction:

Improvebias feedbackVSAvoidpost-processing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent replaces manual post-processing review with an automated bias estimation model that uses machine learning to evaluate bias in generated text. This substitution maintains reliability through bias feedback while significantly reducing the time required compared to manual post-processing.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12505314B2Apparatus and method for outputting language model from which bias has been removed

Publication Date: 2025.12.23 ELECTRONICS & TELECOMM RES INST

AI summary

The present disclosure relates to an apparatus for outputting a language model from which a bias has been removed. The apparatus according to the present disclosure includes a bias estimation model configured to estimate a bias of text to be generated, a bias determination unit configured to determine a bias of next text to be generated through the bias estimation model, and a deep learning-based text generation model configured to generate the next text based on a result of the determination of the bias of the next text.