Logging Schema for Service Metric Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In collaborative services, tracking and aggregating large volumes of data requests across multiple subsystems to evaluate system performance and reliability is complex, making it difficult to extract valuable metrics for consistent monitoring.
Innovation Solution
A logging schema is employed to track user requests, creating entries with subsystem information, operation details, start and end times, and error detection, enabling continuous monitoring of performance and reliability by categorizing errors into buckets for error rate calculation and generating reports.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large volumes of data requests are tracked across multiple subsystems, then monitoring coverage is improved, but data aggregation complexity increases
Solution Approach 1:
The patent segments the complex data aggregation task by introducing standardized logging schemas at each subsystem level. Each subsystem independently logs data according to its specific schema, which is then aggregated centrally. This division of complexity from individual components to centralized aggregation resolves the contradiction by maintaining comprehensive monitoring coverage while managing aggregation complexity through standardization.
Solution Approach 2:
The patent transforms raw data requests into structured log entries with standardized parameters (timestamps, subsystem identifiers, operation types, status codes). This parameter transformation enables consistent aggregation across diverse subsystems, improving monitoring coverage while reducing the complexity of data processing through uniform data structures.
2Difficulty of detecting and measuring
If detailed logging entries are created for each request, then error detection capability is improved, but data storage requirements increase
Solution Approach 1:
The patent extracts only the essential error-related parameters into standardized logging schemas (error codes, error messages, timestamps, subsystem identifiers). By selecting and logging only the critical error detection parameters rather than complete request data, the system improves error detection capability while controlling data storage requirements through selective information extraction.
3Reliability
If continuous monitoring is implemented across all subsystems, then service reliability assessment is improved, but system overhead increases
Solution Approach 1:
The patent implements self-service monitoring where each subsystem automatically generates and logs its own performance and error data according to standardized schemas. This eliminates the need for external monitoring overhead at each subsystem, as components self-report their status. The centralized aggregation then processes these self-generated logs to assess service reliability, improving assessment capability while minimizing system overhead through automated self-monitoring.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Technologies are generally described to provide a passive monitoring system employing a logging schema to track usage data in order to analyze performance and reliability of a service. The logging schema may be configured to track user requests as each request is received and processed at individual subsystems of the collaborative service. A logging entry may be created at a data store of the service, where the logging entry includes a subsystem name, an operation performed by the subsystem to fulfill the request, and start and end times of the operation. The logging schema may also detect errors fulfilling the requests, and may classify detected errors into a bucket, where each bucket denotes a failure scenario. Reliability of the service may be calculated based on analysis of the buckets to compute error rates. Reports may be generated to enable continuous monitoring of a performance and reliability of the system.