Telemetry Filter Routing Using CEL Predicates for Flexible Data Paths
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing telemetry filtering/routing solutions in data sharing platforms lack flexibility to support a wide variety of data paths and custom filtering needs, leading to inefficiencies and computational complexity, and do not allow for optimization of filtering solutions.
Innovation Solution
Implementing user-provided telemetry filtering definitions transpiled into Common Expression Language (CEL) for flexible filtering and routing, enabling customization and optimization of filtering operations across different components of the data sharing platform.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing telemetry filtering/routing solutions are used, then implementation is straightforward, but flexibility to support diverse data paths and custom filtering needs is limited
Solution Approach 1:
The patent introduces Common Expression Language (CEL) as an intermediary representation layer between user-defined filter predicates and the telemetry filtering/routing execution. User filters are transpiled into CEL expressions, which are then evaluated by a CEL interpreter. This intermediary approach enables flexible customization of filtering logic while maintaining efficient execution through standardized evaluation, resolving the contradiction between adaptability and complexity.
Solution Approach 2:
The patent transforms filter predicates into a different parameter representation format (CEL expressions) that can be universally evaluated across diverse data paths. By changing the representation parameters of filter definitions from proprietary formats to a standardized expression language, the system achieves versatility in supporting custom filtering needs without proportionally increasing computational complexity.
2Adaptability or versatility
If user-provided custom filters are implemented, then filtering flexibility is improved, but computational complexity increases
Solution Approach 1:
CEL serves as an intermediary that standardizes user-provided custom filters into a uniform evaluation framework. The transpilation process converts diverse user filter definitions into CEL expressions, which are then efficiently evaluated by a dedicated interpreter. This approach maintains custom filtering flexibility while controlling computational complexity through optimized expression evaluation.
Solution Approach 2:
The patent creates a copy of the filter logic in the form of CEL expressions that can be pre-compiled and cached. Instead of interpreting raw user filters repeatedly, the system compiles them into CEL representation once and reuses the compiled form for multiple evaluations, reducing computational complexity while preserving filtering flexibility.
3Productivity
If telemetry data is filtered early in the pipeline, then downstream processing efficiency is improved, but filtering accuracy may be compromised without full data context
Solution Approach 1:
The patent implements dynamic filtering where CEL expressions are evaluated at multiple stages of the telemetry pipeline based on data availability. Filters can be applied early when possible for efficiency, and refined later when more context is available. This dynamic approach allows the system to adapt filtering depth and timing, achieving both downstream efficiency and filtering accuracy.
Solution Approach 2:
The filtering process is segmented into multiple evaluation stages along the telemetry pipeline. Different portions of the filter logic can be executed at different stages depending on data availability. This segmentation allows early filtering for efficiency while preserving the option for more accurate later filtering, resolving the contradiction between productivity and measurement precision.
Data Source
AI summary
Disclosed are techniques for routing and filtering telemetry data based on custom telemetry definitions provided by a user. A telemetry filter definition comprising rules for routing and filtering telemetry data may be converted into a common expression language (CEL) abstract syntax tree (AST). The CEL AST may be provided to a filtering component, which may compile the CEL AST into a CEL filter program comprising the rules for routing and filtering telemetry data. In response to receiving telemetry data, filtering, by the filtering component, the received telemetry data based on the CEL filter program to generate filtered telemetry data.