Hierarchical Concept Tree Construction from Table Corpora

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing natural language-based techniques for knowledge discovery in enterprise domains face challenges due to scarcity of textual data and inconsistencies in entity representations, making it difficult to apply and scale effectively within private networks.

Innovation Solution

A dynamic programming approach is used to construct a hierarchical concept tree from a large table corpus, employing statistical co-occurrence values and a parallel algorithm to efficiently cluster and reduce the tree, addressing the scalability and compatibility issues in enterprise domains.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If natural language-based techniques are applied to enterprise domains, then knowledge discovery can be performed, but the scarcity of textual data reduces reliability

Engineering Contradiction:

Improveknowledge discovery reliabilityVSAvoidtextual data quantity

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent introduces table data as an intermediary resource between the scarcity of textual documents and the need for knowledge discovery. Instead of relying solely on natural language texts, the system extracts and processes structured table data from enterprise systems, using it as a mediator to build concept trees and discover knowledge in domains where textual data is limited.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Loss of information

If natural language techniques are used to discover entity concepts, then knowledge can be extracted, but inconsistencies in entity representations require substantial post-processing

Engineering Contradiction:

Improveknowledge extraction completenessVSAvoidpost-processing complexity

Core Design Contradiction:

Loss of informationVSDevice complexity

Solution Approach 1:

The patent segments the knowledge discovery process into distinct phases: table data extraction, concept identification from tables, hierarchy construction, and concept tree formation. By segmenting the process and handling each phase separately with appropriate methods, the system reduces the need for extensive post-processing cleanup that would be required if using monolithic natural language processing approaches.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the fundamental parameter of data representation from unstructured natural language text to structured table data with defined schemas. This parameter change transforms inconsistent entity mentions into standardized table rows and columns, inherently reducing representation inconsistencies and the associated post-processing burden.

Inventive Principle:

Principle #35Parameter changes

3Adaptability or versatility

If enterprise-specific proprietary data is processed, then domain-specific knowledge can be discovered, but scalability across different enterprises becomes difficult and expensive

Engineering Contradiction:

Improvedomain-specific knowledge discoveryVSAvoidscalability

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent creates a universal framework that can process different enterprise-specific proprietary data formats through a common table extraction and concept tree construction pipeline. The system is designed to work with various data sources (databases, spreadsheets, enterprise systems) by converting them to a standardized table format, enabling scalable deployment across different enterprises without requiring enterprise-specific customization of the core methodology.

Inventive Principle:

Principle #6Universality (Multi-functionality)

4Stability of the object's composition

If concept trees are constructed from table corpora, then coherent concepts can be organized, but computational complexity increases with data size

Engineering Contradiction:

Improveconcept coherenceVSAvoidcomputational complexity

Core Design Contradiction:

Stability of the object's compositionVSDevice complexity

Solution Approach 1:

The patent performs preliminary actions by pre-processing table data into a standardized format, pre-identifying potential concepts and their relationships before constructing the full concept tree. This preliminary organization of data and relationships reduces the computational complexity of the subsequent tree construction phase, as the raw work of data cleaning and relationship identification has already been performed.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10789229B2Determining a hierarchical concept tree using a large corpus of table values

Publication Date: 2020.09.29 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

A table corpus processing server identifies concepts within enterprise domain data. The table corpus processing server is configured to iteratively group values in a table corpus based on co-occurrence statistics to produce a candidate hierarchical tree. The candidate hierarchical tree is then summarized by selecting nodes that can best “describe” the original corpus, which leads to a small tree that often corresponds to desired concept hierarchies. The table corpus processing server employs a parallel dynamic programming approach that allows the disclosed embodiments to scale with amount of enterprise domain data being analyzed.