Knowledge Graph Update from Multi-Modal Sources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The construction and update of knowledge graphs from multi-modal sources, such as historical data and customer service conversations, are inefficient and require manual labeling of questions and answers, making it tedious and costly for applications like chatbots.
Innovation Solution
An apparatus and method that automatically generate and update knowledge graphs using a speaker diarization module, audio transcription, speech parsing, conversation parsing, and a knowledge graph container, which classify speakers, transcribe audio, extract entities and relations, label words, generate question-answer pairs, and update the knowledge graph without manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling of questions and answers is used to construct knowledge graphs from multi-modal data, then the knowledge graph can be built with human expertise and control, but the process becomes tedious and inefficient
Solution Approach 1:
The system automatically extracts questions and answers from multi-modal data sources including customer service conversations and instruction manuals without requiring manual human labeling. The automated extraction process analyzes audio, text, and metadata to generate knowledge graph entries independently, eliminating the tedious manual work while maintaining quality through structured processing pipelines
Solution Approach 2:
The manual mechanical process of human experts labeling questions and answers is replaced with automated computational systems including speech-to-text conversion, natural language processing, and machine learning models that can process multi-modal data at scale, dramatically improving construction efficiency while maintaining systematic quality control
2Reliability
If manual extraction of information from multi-modal sources is performed, then accurate questions and answers can be identified, but the process requires significant time and resources
Solution Approach 1:
The system performs preliminary processing of multi-modal data by converting audio to text, organizing customer service conversations, and pre-processing instruction manuals before extraction. This preliminary structuring of data enables subsequent automated extraction to proceed efficiently without requiring time-consuming manual review of raw unprocessed data
Solution Approach 2:
The time-consuming manual extraction process is replaced with automated information extraction systems that use natural language processing and machine learning to identify and extract questions and answers from multi-modal sources, reducing extraction time from days or weeks to minutes or hours while maintaining reliability through systematic processing
3Measurement precision
If experienced engineers manually prepare questions and answers for knowledge graphs, then the knowledge base can be populated with domain expertise, but the process is costly and inefficient
Solution Approach 1:
The system automatically processes customer service conversations and technical documentation to extract domain-specific questions and answers without requiring experienced engineers to manually create each entry. The automated system maintains domain expertise by learning from existing knowledge bases and applying structured extraction methods to new data
Solution Approach 2:
The costly manual process of experienced engineers preparing knowledge graph entries is replaced with automated natural language processing and machine learning systems that can process large volumes of multi-modal data at fraction of the cost, while maintaining domain expertise through training on existing expert-generated content and systematic extraction from domain-specific sources
Data Source
AI summary
The present invention provides an apparatus and method for automatic generation and update of a knowledge graph from multi-modal sources. The apparatus comprises a conversation parsing module configured for updating a dynamic information word set VD with labelled words generated from extracted from the multi-modal sources; updating a static information word set VS based on extracted schema of relations extracted from the multi-modal sources; and generating pairs of question and answer based on the dynamic information word set VD, the static information word set VS and the one or more sentence patterns; and a knowledge graph container configured for updating a knowledge graph based on the extracted entities of interest and schema of relations. Therefore, an efficient and cost-effective way for question decomposition, query chain construction and entity association from unstructured data is achieved.