Hardware-Accelerated Metadata Generation Using Coprocessors

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional indexing techniques are computationally intensive and time-consuming, making it impractical for enterprises to effectively index large volumes of unstructured data, which constitutes the bulk of their data sets, leading to inefficient data management and search capabilities.

Innovation Solution

The use of hardware-accelerated metadata generation through coprocessors, specifically reconfigurable logic devices like FPGAs, to stream data and generate metadata at bus bandwidth rates, enabling rapid indexing and search capabilities for both structured and unstructured data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If conventional indexing techniques are used, then data can be indexed with standard processing resources, but indexing latency is high and processing speed is slow

Engineering Contradiction:

Improveindexing speedVSAvoidindexing latency

Core Design Contradiction:

SpeedVSLoss of time

Solution Approach 1:

The patent replaces conventional software-based indexing mechanisms with hardware-accelerated coprocessors that perform metadata generation and indexing operations in parallel, significantly increasing processing speed and reducing indexing latency through dedicated hardware circuits optimized for these specific tasks

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent divides the indexing system into separate functional components: main processors that handle high-level coordination and coprocessors that handle specific metadata generation tasks. This segmentation allows parallel processing of different data streams and reduces bottlenecks in the indexing pipeline

Inventive Principle:

Principle #1Segmentation

2Productivity

If hardware-accelerated metadata generation is implemented, then indexing speed increases and latency reduces, but system complexity increases

Engineering Contradiction:

Improveindexing throughputVSAvoidsystem architecture complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent introduces coprocessors as intermediary hardware components that bridge the gap between main processors and data storage systems. These coprocessors handle the computationally intensive metadata generation tasks, allowing main processors to focus on coordination and data management, thereby increasing overall productivity while managing complexity through clear functional separation

Inventive Principle:

Principle #24Intermediary (Mediator)

3Reliability

If more processing resources are allocated to indexing, then indexing performance improves, but costs increase

Engineering Contradiction:

Improvesearch capabilityVSAvoidhardware resources

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent changes the physical state of processing by transitioning from software-based indexing to hardware-accelerated indexing using coprocessors. This parameter change enables significantly higher indexing throughput and more reliable search capabilities while using a moderate increase in hardware resources, as the coprocessors are specifically optimized for metadata generation tasks

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentEP2092419B1Method and system for high performance data metatagging and data indexing using coprocessors

Publication Date: 2020.01.01 IP RESERVOIR LLC

Figure 1
Figure 2(a)
Figure 2(b)

AI summary

Disclosed herein is a method and system for hardware-accelerating the generation of metadata for a data stream using a coprocessor (450). Using these techniques, data can be richly indexed, classified, and clustered at high speeds. Reconfigurable logic (402) such a field programmable gate arrays (FPGAs) can be used by the coprocessor for this hardware acceleration. Techniques such as exact matching, approximate matching, and regular expression pattern matching can be employed by the coprocessor to generate desired metadata for the data stream.