Extended Telemetry Metrics for Structured UDF Event Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing solutions for generating and managing telemetry data from User-Defined Functions (UDFs) in secure environments are not scalable and lack standardization, leading to unstructured log data and challenging search/query processes.
Innovation Solution
Implementing managed event tables and extended telemetry metric data types using the OpenTelemetry framework to structure telemetry data, allowing users to easily query and analyze trace events and metrics from UDFs executed in a sandbox environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing solutions are used for generating telemetry data from UDFs in secure environments, then implementation is simpler, but the system lacks scalability and standardization leading to unstructured log data
Solution Approach 1:
The patent introduces extended metric data types (string, boolean, binary, timestamp, array, object) beyond traditional numeric types, fundamentally changing the parameter space of telemetry data representation. This enables structured logging with rich data types while maintaining compatibility with existing OpenTelemetry frameworks through systematic type extension rather than complete system redesign
Solution Approach 2:
The patent segments the telemetry data structure into distinct components: metric name, metric type, data type, unit, description, and extended type-specific fields (e.g., string_value, boolean_value, timestamp_value). This segmentation allows each data type to be handled independently with appropriate serialization while contributing to a unified structured logging system
2Manufacturing precision
If extended telemetry metric data types are implemented, then structured telemetry data generation is improved, but data type complexity increases
Solution Approach 1:
The patent creates a universal metric data structure that can accommodate multiple data types (numeric, string, boolean, binary, timestamp, array, object) through a single unified schema. The MetricData class uses type discrimination to route to appropriate value representations, allowing one structure to serve multiple data type functions without requiring separate structures for each type
Solution Approach 2:
The patent introduces an intermediary type system that sits between the raw UDF log data and the final structured telemetry output. The extended data types act as mediators that standardize diverse data formats into a common structure that can be processed by existing OpenTelemetry pipelines, bridging the gap between complex source data and standardized output
3Ease of operation
If unstructured log data is generated from UDFs, then implementation is easier, but search and query processes become challenging
Solution Approach 1:
The patent changes the fundamental parameters of log data from unstructured strings to structured objects with typed fields. Each metric is represented with explicit data types, units, and semantic meaning, transforming information that would be lost in unstructured formatting into preserved, queryable structured data that maintains full semantic content
Solution Approach 2:
The patent effectively 'colors' or tags each piece of telemetry data with metadata including data type, unit, description, and type-specific markers (e.g., 'STRING_TYPE_', 'TIMESTAMP_TYPE_'). These semantic tags enable efficient filtering, searching, and querying by data characteristics without losing the underlying information, making data easily distinguishable and searchable
Data Source
AI summary
Provided herein are systems and methods for configuring extended telemetry metric data types. A system includes at least one hardware processor coupled to a memory and configured to detect an API call at an execution node of a database system. Metric data emitted by a telemetry API corresponding to the API call is collected. The metric data is converted from a first data format associated with the telemetry API to a second data format to generate formatted metric data. The second data format is a native format to an event table of the database system. Telemetry data based on the formatted metric data is ingested into the event table.