BERT Attention Highlighting for Question Answer Reliability

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing question-answer pair searching systems struggle to effectively highlight important words in answers, making it difficult for users to quickly understand and trust the reliability of the answers.

Innovation Solution

The system uses a BERT model to generate attention matrices, which are then used to calculate attribution values for each word in the answer. These attribution values determine which words to highlight, allowing users to quickly identify important information and trust the model's accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If answers are presented in full text without highlighting, then complete information is provided, but user understanding and identification of important information becomes difficult

Engineering Contradiction:

Improvecompleteness of answer informationVSAvoiduser understanding and identification of important information

Core Design Contradiction:

Loss of informationVSEase of operation

Solution Approach 1:

The patent applies local quality by highlighting specific words and phrases within the answer text based on their attribution values. Instead of treating the entire answer uniformly, the system identifies and visually distinguishes important portions (through highlighting) while preserving the complete answer text, thus simultaneously maintaining information completeness and improving user understanding.

Inventive Principle:

Principle #3Local quality

2Measurement precision

If gradient-based attribution methods are used to determine important words, then accurate attribution is achieved, but computational cost becomes excessively high

Engineering Contradiction:

Improveaccuracy of word attributionVSAvoidcomputational cost and energy consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent replaces the computationally expensive gradient-based attribution method with a more efficient alternative that uses pre-computed attention weights from the transformer model. This substitution maintains reasonable attribution accuracy while dramatically reducing computational cost and energy consumption, making the system feasible for practical deployment.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

3Loss of information

If long spans of text are presented in answers, then comprehensive information is provided, but user confidence in model reliability decreases

Engineering Contradiction:

Improvecomprehensiveness of answerVSAvoiduser confidence in model accuracy

Core Design Contradiction:

Loss of informationVSReliability

Solution Approach 1:

The patent uses visual highlighting (color changes) to distinguish important words and phrases within the answer text. This visual differentiation helps users quickly identify the model's key conclusions and reasoning, thereby increasing user confidence in the model's reliability while maintaining the comprehensive text of the answer.

Inventive Principle:

Principle #32Color changes

Data Source

PatentUS20250139146A1Systems and methods for highlighting key words and phrases from answers

Publication Date: 2025.05.01 RELX INC

AI summary

In embodiment, a method of displaying an answer of a question-answer pair in response to a natural language search query includes receiving, from a Bidirectional Encoder Representations from Transformers (BERT) model, an array of attention matrices for the question-answer pair, where each attention matrix of the array of attention matrices includes an array of attribution values, generating a total attribution value for each word of an answer of the question-answer pair from the array of attention matrices, and displaying the answer on an electronic display, wherein one or more words of the answer is highlighted based on the total attribution values for each word.