Relation Discovery System for Multi-Cloud Log Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In hybrid and multi-cloud computing environments, existing technologies face challenges in efficiently monitoring and managing relations between operation objects across multiple levels, leading to suboptimal network operations, storage utilization, and network throughput.
Innovation Solution
A method and system for relation discovery from operation data, which classifies extracted entities into categories, determines log affiliation, identifies and builds relations, and performs statistical correlation analysis to remove redundancy, enhancing relation mapping and knowledge base updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If relation discovery is performed across multiple logs in hybrid/multi-cloud environments, then monitoring coverage and relation mapping accuracy are improved, but data processing complexity and computational resources required increase
Solution Approach 1:
The system segments the complex task of relation discovery across multiple logs by processing logs in batches and grouping entities by their log affiliations. The correlation discovery module operates on specific log segments at a time, reducing the complexity of analyzing the entire multi-cloud environment simultaneously while maintaining comprehensive monitoring coverage.
Solution Approach 2:
The system introduces intermediate data structures including entity graphs, relation graphs, and standardized schemas that act as mediators between raw operation data and final relation mappings. These intermediaries simplify the processing complexity by providing structured representations that can be efficiently queried and correlated across different logs.
2Quantity of substance
If statistical correlation analysis is performed to remove redundancy, then data quality and storage efficiency are improved, but processing time and computational overhead increase
Solution Approach 1:
The system performs preliminary actions by pre-processing operation data to extract and classify entities before the main correlation analysis. Entities are grouped by log affiliation and pre-organized in data structures that enable efficient redundant detection during the correlation phase, reducing the overall processing time while maintaining data quality.
Solution Approach 2:
The system changes parameters such as correlation thresholds, statistical significance levels, and redundancy removal criteria to optimize the balance between processing time and data quality. By dynamically adjusting these parameters based on the specific multi-cloud environment and data characteristics, the system achieves efficient redundant removal without excessive computational overhead.
3Reliability
If comprehensive relation mapping is performed across all operation objects, then monitoring completeness is improved, but system response time and throughput decrease
Solution Approach 1:
The system applies partial action by focusing correlation analysis on the most relevant log combinations and entity types first, rather than uniformly processing all possible relations. The correlation discovery module prioritizes relations based on predefined importance weights and operational context, achieving monitoring completeness for critical components while maintaining high system throughput.
Solution Approach 2:
The system implements dynamic relation mapping that adapts to changing operational conditions. The correlation discovery module adjusts its processing scope and depth based on real-time metrics such as data volume, complexity levels, and operational priorities, allowing the system to maintain both completeness and throughput by shifting focus between comprehensive analysis and targeted processing.
Data Source
AI summary
A method and system for relation discovery from operation data includes classifying categories of extracted entities from operation data into three or more classes identified in a knowledge base. A log affiliation of the extracted entities is determined, and relations of the extracted entities are identified according to a log affiliation. The identified relations information of the extracted entities is associated with operation objects of the operation data.