Telemetry Format Conversion Library for Heterogeneous Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current solutions face challenges in converting raw telemetry data from operational systems into a normalized data format, particularly due to heterogeneity in data formats and the complexity of configuring data sources and additional conversion components.
Innovation Solution
An apparatus comprising a processor and memory configured to receive raw data units and format conversion routing metadata, select appropriate format conversion functions, and execute these functions to convert raw data into a normalized format, allowing data to be transmitted in its original format and normalized later.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If raw data is converted to normalized format immediately at the source, then data consistency is improved, but device complexity and processing overhead increase
Solution Approach 1:
The patent applies preliminary action by pre-defining and storing format conversion functions in a library before data arrives. When raw data is received, the system only needs to select and execute the appropriate pre-defined function, avoiding the complexity of creating conversion components on-demand. This resolves the contradiction by preparing conversion logic in advance while maintaining simplicity during actual data processing.
Solution Approach 2:
The patent introduces an intermediary format conversion function library that sits between the raw data sources and the normalized data consumers. This intermediary layer handles the format conversion centrally, allowing data sources to remain simple while providing normalized output to consumers. The library acts as a mediator that resolves format heterogeneity without requiring changes at either the source or consumer ends.
2Adaptability or versatility
If multiple format conversion components are configured for different data sources, then data format compatibility is improved, but ease of operation deteriorates
Solution Approach 1:
The patent applies universality by creating a single format conversion function library that handles multiple different data formats through a unified interface. Instead of requiring separate conversion components for each data source, the library provides universal conversion capabilities where functions are selected based on the data source type. This maintains broad format compatibility while simplifying operation to a single centralized system.
Solution Approach 2:
The patent uses copying by replicating conversion logic as reusable functions within the library. Each format conversion function is a copy of the conversion algorithm, stored and reused when needed. This allows the system to handle multiple formats without requiring unique conversion components for each, reducing configuration complexity while maintaining adaptability to various data formats.
3Productivity
If data conversion is performed in real-time at the point of receipt, then data processing efficiency is improved, but loss of time for conversion operations increases
Solution Approach 1:
The patent applies preliminary action by pre-compiling and storing format conversion functions in a library before data arrives. When raw data is received, the system only needs to select and execute the appropriate pre-defined function rather than creating conversion logic on-demand. This resolves the contradiction by preparing conversion logic in advance, enabling fast real-time execution without the time overhead of dynamic code generation or component instantiation.
Data Source
AI summary
A format conversion system provides an extendable library of data format conversion functions configured to convert raw data (e.g., telemetry data) from any number of original formats to at least one normalized format. Received raw data is routed to the appropriate format conversion function in the library based on format conversion routing metadata generated with respect to raw data from a particular source and/or a mapping of data sources to corresponding original data formats and/or to corresponding format conversion functions. A selected format conversion function from the library is executed with respect to raw data from particular data sources, and normalized data corresponding to the raw data is output. A format conversion function definition interface enables user configuration of the various format conversion functions, and a configuration interface enables user configuration of the mapping used to route the raw data to the various format conversion functions.