Automated Text Metadata Extraction via Feature Clustering

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current methods for annotating unlabelled text documents are manual, time-consuming, error-prone, and inadequate for processing large-scale 'big data', lacking an automated approach to determine meta data such as concepts and content information.

Innovation Solution

A method involving the creation of a feature matrix from text documents, clustering using algorithms like Non-Negative Matrix Factorization (NMF), scoring, and ranking features to output meta data sets, enabling automated and efficient processing of unlabelled text documents without requiring expert knowledge.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If manual annotation by experts is used to determine meta data for text documents, then the accuracy and reliability of meta data extraction is improved, but the time consumption and cost increase significantly

Engineering Contradiction:

Improveaccuracy of meta data extractionVSAvoidtime consumption for annotation

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system enables self-service by allowing the text documents to automatically generate their own meta data through the automated processing pipeline. The feature extraction, clustering, and ranking algorithms work autonomously without requiring expert intervention, making the system serve itself in extracting meaningful information from the documents.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces the mechanical manual annotation process with an automated computational system. Instead of experts manually reading and annotating documents, the system uses feature extraction, matrix factorization, and algorithmic ranking to automatically determine meta data, substituting human mechanical work with automated processing.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Reliability

If manual annotation by experts is used to determine meta data for text documents, then the quality of meta data is improved, but the error rate decreases only at the cost of increased resource consumption

Engineering Contradiction:

Improvequality of meta dataVSAvoidprocessing capacity for large data sets

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system processes large data sets autonomously without requiring expert resources. The automated pipeline of feature extraction, clustering, and ranking enables the system to handle voluminous documents independently, eliminating the need for proportional increases in expert annotators as data volume grows.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent transforms the unlabelled text documents into a structured feature matrix, changing the parameter representation from raw text to numerical features. This parameter transformation enables automated processing while maintaining the quality of meta data extraction through systematic analysis of feature relationships.

Inventive Principle:

Principle #35Parameter changes

3Device complexity

If traditional data processing methods are used, then the simplicity of the process is maintained, but the ability to handle big data adequately is insufficient

Engineering Contradiction:

Improvesimplicity of processing methodVSAvoidability to process large scale data

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The patent segments the complex task of meta data extraction into distinct processing stages: feature extraction, matrix factorization, clustering, and ranking. This segmentation makes the overall complex process manageable and adaptable to large data sets while maintaining clear procedural structure.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system transitions from processing text documents in one dimension to representing them in multiple dimensions through the feature matrix and clustering space. This dimensional transformation enables the system to capture complex relationships in the data while maintaining computational tractability through structured mathematical operations.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS11263251B2Method for determining output data for a plurality of text documents

Publication Date: 2022.03.01 SIEMENS AG

AI summary

Provided is a method for determining output data for a plurality of text documents, including the steps of: providing a feature matrix as input data; wherein the feature matrix includes information about frequencies of a plurality of features within the plurality of text documents; clustering the feature matrix using a clustering algorithm into at least one clustering matrix; wherein the at least one clustering matrix includes information about the cluster membership of each document of the plurality of documents or each feature of the plurality of features, assigning at least one score to each feature of the plurality of features based on the at least one clustering matrix; ranking the plurality of features based on their assigned scores; and outputting the ranked features as output data. A corresponding computer program product and system is also provided.