Database Observability Querying via Trigger Objects

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current database systems face inefficiencies in accessing and managing large volumes of observability data, leading to delayed identification and remediation of database-related incidents due to data being stored in various formats and locations, with limited access permissions and short data retention periods.

Innovation Solution

A system that utilizes a trigger object to specify data retrieval parameters, allowing for on-demand access to database observability data by generating queries and aggregating data into an external event store, enabling quick and organized access to necessary information for administrators.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If continuous collection and storage of granular observability data is maintained, then data availability for incident analysis is improved, but storage cost and resource consumption increase significantly

Engineering Contradiction:

Improveobservability data availabilityVSAvoidstorage resource consumption

Core Design Contradiction:

Loss of informationVSQuantity of substance

Solution Approach 1:

The patent extracts only the essential observability data needed for incident analysis by using predefined query templates that specify which metrics and logs are relevant. Instead of storing all granular data, the system extracts and stores only the data that matches these templates, significantly reducing storage requirements while maintaining data availability for troubleshooting.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The system changes the parameter of data retention by implementing time-based retention policies where observability data is kept for specific durations based on its type and importance. Less critical data is retained for shorter periods, while critical incident-related data is retained longer, optimizing the balance between data availability and storage resource consumption.

Inventive Principle:

Principle #35Parameter changes

2Reliability

If access to observability data is restricted to highly privileged users only, then security is improved, but ease of operation for support teams deteriorates

Engineering Contradiction:

ImprovesecurityVSAvoiddata access convenience

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The patent introduces an intermediary layer in the form of a data lake and query interface that sits between the observability data sources and support team members. This intermediary allows support staff to access and query observability data without needing high-level privileges, as the system handles authentication and authorization through the intermediary layer while maintaining security protocols.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system implements a universal query interface that serves multiple functions: it provides secure access control, enables data querying for support teams, maintains audit trails, and enforces retention policies. This multi-functional interface allows various user roles to access appropriate data without requiring individual privilege management for each data source.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Productivity

If observability data is deleted after a short retention period, then storage efficiency is improved, but ability to diagnose historical incidents deteriorates

Engineering Contradiction:

Improvestorage efficiencyVSAvoidincident diagnosis time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system implements dynamic retention parameters where different types of observability data are retained for different durations based on their diagnostic value. Critical metrics and logs related to incident-prone systems are retained longer, while routine operational data is retained for shorter periods. This parameter-based retention strategy optimizes both storage efficiency and incident diagnosis capability.

Inventive Principle:

Principle #35Parameter changes

4Quantity of substance

If on-demand data retrieval is implemented instead of continuous storage, then resource consumption is reduced, but data access latency may increase

Engineering Contradiction:

Improveresource consumptionVSAvoiddata access speed

Core Design Contradiction:

Quantity of substanceVSSpeed

Solution Approach 1:

The patent implements preliminary action by pre-defining query templates and data retrieval patterns based on common incident scenarios. When an incident occurs, the system can quickly execute these pre-configured queries without needing to design and optimize queries in real-time, thus maintaining fast access speeds while using on-demand retrieval instead of continuous storage.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12124458B2Database system observability data querying and access

Publication Date: 2024.10.22 SALESFORCE INC

AI summary

A query may be determined based on data retrieval parameter values. The data retrieval parameter values may include a data type parameter identifying a designated data type of a plurality of data types corresponding with database observability data generated during operation of a database system. The of data retrieval parameter values may also include a designated time period during which to retrieve data associated with the designated data type. A query may be determined based on the plurality of data retrieval parameter values via a collection manager in a database application layer associated with the database system. The query may be executed to retrieve a designated dataset of database observability data corresponding with the designated data type and generated by the database system during the designated time period.