Knowledge-Based Deep Learning Refactoring with Structured Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning models face unsustainable growth in size due to incorporation of unstructured information, leading to inefficiencies, limited extensibility, and challenges in Out-Of-Distribution accuracy, domain adaptation, and explainability.
Innovation Solution
A Knowledge-Based Refactored Deep Learning Model (KBRM) separates functional skills in a Neural Network from structured Knowledge Base information, allowing per-query retrieval of needed data, reducing model size and enhancing extensibility, explainability, and domain adaptability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If unstructured information is incorporated into deep learning models, then model knowledge coverage is improved, but model size grows unsustainably
Solution Approach 1:
The patent segments the deep learning model into two distinct components: a compact neural network module that handles functional processing and a separate knowledge base that stores structured information. This segmentation allows the model to maintain small size while accessing extensive knowledge through structured storage external to the neural network parameters.
Solution Approach 2:
The patent extracts structured knowledge from unstructured information sources and stores it in a separate knowledge base, removing it from the neural network parameters. This extraction process reduces model size while preserving knowledge accessibility through structured queries and retrieval mechanisms.
2Measurement precision
If more information is stored in the model, then answer accuracy is improved, but retrieval efficiency deteriorates
Solution Approach 1:
The patent applies local quality by organizing knowledge in the knowledge base with different structural properties optimized for specific retrieval needs. Structured knowledge is organized with metadata, relationships, and indexing that enable efficient queries for specific types of information, allowing fast retrieval without compromising answer accuracy.
3Quantity of substance
If model size increases, then knowledge capacity is improved, but extensibility and domain adaptability worsen
Solution Approach 1:
The patent implements dynamics by making the knowledge base externally accessible and modifiable without retraining the neural network. New domains and knowledge can be dynamically added to the knowledge base in structured format, allowing the model to adapt to new domains quickly by simply adding structured knowledge rather than retraining entire model parameters.
4Adaptability or versatility
If unstructured information is used, then model comprehensiveness is improved, but explainability deteriorates
Solution Approach 1:
The patent introduces structured knowledge as an intermediary between unstructured information sources and the neural network. This structured representation serves as a mediator that organizes information in a traceable, queryable format, enabling the system to maintain comprehensiveness while providing explainability through structured knowledge paths and retrieval traces.
Data Source
AI summary
Methods and apparatus for a knowledge-based deep learning refactoring model with tightly integrated functional nonparametric memory are disclosed. An example non-transitory computer readable medium comprises instructions that, when executed, cause a machine to at least estimate a first information extraction cost corresponding to retrieval of information from a local knowledge base, estimate a second information extraction cost corresponding retrieval of information from a remote knowledge base, select an information source based on the first and second estimated information extraction costs, query the selected information source, in response to determining that the selected information source was an external information source, store the queried information in the local knowledge base, organize the stored information in the local knowledge base, and return the queried information.