Log Storage Segmentation by Traffic Type for Cloud Microservices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The efficiency of log storage systems is low due to the large quantity of log information generated by numerous microservices in cloud service systems.
Innovation Solution
Determine log servers and storage paths based on service identifiers and traffic types to separate high-traffic and low-traffic services, implementing a computing and storage architecture that optimizes log information writing and querying.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If all log information from numerous microservices is stored in a single log storage system, then complete log data is preserved, but storage efficiency deteriorates due to the large quantity of log information
Solution Approach 1:
The patent segments log information storage by creating multiple independent log servers grouped into different log server groups. Each group stores logs for specific traffic types (e.g., high-traffic services vs. low-traffic services), dividing the monolithic storage system into manageable segments that can operate independently, thereby improving overall storage efficiency.
Solution Approach 2:
The patent introduces a new dimension of organization by classifying log servers into groups based on traffic types. This dimensional classification (high-traffic group, medium-traffic group, low-traffic group) adds a hierarchical structure to the storage system, allowing logs to be routed to appropriate groups based on their traffic characteristics, thus improving storage efficiency.
2Loss of information
If log information is centralized in one storage system, then data completeness is maintained, but querying efficiency deteriorates due to large search ranges
Solution Approach 1:
The patent segments the log storage system into multiple log server groups, each handling specific traffic types. This segmentation allows query operations to be confined to relevant groups only, reducing the search range and querying time while maintaining complete log data across all groups.
Solution Approach 2:
The patent applies local quality by optimizing storage and query handling for different traffic types in different log server groups. High-traffic service logs are stored in groups optimized for fast writing and querying, while low-traffic service logs are stored in groups with different optimization characteristics, ensuring each group has the quality needed for its specific purpose.
3Device complexity
If a single log server handles all services, then system complexity is low, but processing efficiency deteriorates due to traffic variations
Solution Approach 1:
The patent implements dynamics by creating multiple log server groups that can be dynamically selected based on traffic type. The system adapts to varying traffic conditions by routing different traffic types to appropriate log server groups, improving processing efficiency without requiring each individual server to handle all traffic types simultaneously.
Solution Approach 2:
The patent applies universality by designing log server groups that can handle different traffic types. Each log server group is capable of processing logs for services within its traffic type category, creating a multi-functional system where groups can be selectively activated based on the specific logging needs, thereby improving overall processing efficiency.
Data Source
AI summary
A method for processing a log request includes: obtaining, for a write request for log information of each service, a service identifier of the service and a traffic type of the service, where the traffic type is used to indicate a write traffic range of the log information of the service within a preset duration; determining, based on the service identifier and the traffic type of the service, at least one target log server that processes the write request, and determining, based on the service identifier and the traffic type of the service, storage path information of the log information of the service on a log storage unit; and writing, through the at least one target log server, the log information of the service into the log storage unit based on the storage path information.