Knowledge Graph Assisted LLM Chatbots for Reliable Novel Queries

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing chatbots, whether rule-based or AI-driven, struggle with handling complex and unexpected queries due to limitations in understanding context and generating accurate responses, while knowledge graphs are limited in handling novel queries and require extensive datasets.

Innovation Solution

A knowledge-graph system that integrates with large language models (LLMs) to build and augment knowledge graphs, using LLM answers to provide accurate responses, store provenance information, and validate answers, thereby leveraging the strengths of both systems.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If LLMs are used to generate responses to queries, then the ability to handle complex and novel queries is improved, but the accuracy and reliability of responses deteriorate due to hallucinations and inaccuracies

Engineering Contradiction:

Improveability to handle complex and novel queriesVSAvoidaccuracy of responses

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent introduces a knowledge graph as an intermediary component between the user query and the LLM. The knowledge graph stores verified factual information and relationships, serving as a mediator that provides accurate, grounded context to the LLM. This allows the LLM to generate responses based on verified knowledge rather than relying solely on its trained parameters, thereby reducing hallucinations while maintaining the ability to handle complex queries.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent merges two previously separate systems - the knowledge graph system and the LLM-based chatbot - into a unified hybrid system. The knowledge graph handles factual verification and structured knowledge retrieval, while the LLM handles natural language understanding and response generation. This combination allows the system to leverage the strengths of both approaches: the reliability and factual accuracy of knowledge graphs and the flexibility and contextual understanding of LLMs.

Inventive Principle:

Principle #5Merging (Combining)

2Reliability

If knowledge graphs are used to store and retrieve information, then the reliability and factual accuracy of responses are improved, but the ability to handle novel and complex queries deteriorates due to limited coverage

Engineering Contradiction:

Improvefactual accuracy of responsesVSAvoidability to handle novel and complex queries

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The LLM acts as an intermediary that bridges the gap between the structured knowledge graph and novel, complex queries. When a query is received, the LLM first processes the natural language input, identifies relevant concepts and relationships, and then queries the knowledge graph for verified factual information. This allows the system to handle novel queries by leveraging the LLM's contextual understanding while grounding the response in verified knowledge from the knowledge graph.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The hybrid system achieves multi-functionality by combining the capabilities of both knowledge graphs and LLMs. The knowledge graph provides factual accuracy and structured knowledge retrieval, while the LLM provides natural language processing, contextual understanding, and handling of novel scenarios. Together, they create a universal system that can reliably handle both routine factual queries and complex, novel situations.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Productivity

If LLMs are used to build and augment knowledge graphs, then the efficiency of knowledge graph construction is improved, but the accuracy of stored information deteriorates due to potential hallucinations

Engineering Contradiction:

Improveefficiency of knowledge graph constructionVSAvoidaccuracy of stored information

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent implements a feedback mechanism where LLM-generated information is validated against existing knowledge graph data and external sources before being stored. The system continuously monitors and verifies the accuracy of augmented knowledge, using feedback loops to correct errors and improve reliability. This allows efficient knowledge graph construction using LLMs while maintaining high accuracy through systematic verification and validation processes.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS12531820B2Knowledge graph assisted large language models

Publication Date: 2026.01.20 AMAZON TECH INC

AI summary

Techniques for a knowledge-graph system to use large language models (LLMs) to build knowledge graphs to answer queries submitted to a chatbot by users. The knowledge-graph system builds the knowledge graph using answers produced by an LLM for novel queries. The chatbot will continue to use the LLM to answer novel queries, but the chatbot may harness the knowledge graph to answer repeat questions to gain various efficiencies over LLM-backed chatbots. For example, the knowledge-graph system may easily debug or otherwise improve the answers in knowledge graphs, store provenance information in knowledge graphs, and augment the knowledge graphs using other data sources. Thus, the reliability and correctness of chatbots will be improved as the bugs and inaccuracies in answers provided by the LLM will be corrected in the knowledge graphs, but the chatbots can still harness the abilities of LLMs to provide answers across various subject-matter domains.