Log Redaction via Secret Splitting and Entropy Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing logging systems often inadvertently log restricted information, such as passwords and sensitive data, due to less restrictive access controls, leading to potential leaks through log file analysis.
Innovation Solution
Implementing a logging service that registers secrets upon retrieval, splits credentials into portions, and uses entropy-based filtering to redact sensitive information, with a protection window mechanism to manage and reduce the search list, ensuring only partially exposed data is logged, thereby preventing full secret exposure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If logging systems use less restrictive access controls to improve ease of operation, then logging functionality is improved, but sensitive information such as passwords and secrets may be inadvertently logged, leading to security risks
Solution Approach 1:
The system performs preliminary redaction by splitting secrets into portions and registering fingerprints before logging occurs. The logging service proactively identifies and redacts sensitive information from log entries before they are written to log files, preventing security risks while maintaining ease of logging operation
Solution Approach 2:
The patent introduces a logging service as an intermediary component between the application and the log files. This service acts as a mediator that intercepts log entries, performs redaction of sensitive information using fingerprint matching, and then writes the sanitized logs to files, thus resolving the contradiction between easy logging and security
2Reliability
If the logging service maintains a comprehensive search list of all fingerprints to ensure complete redaction coverage, then security is improved, but the complexity and computational overhead of the logging service increases
Solution Approach 1:
The patent segments the secret into multiple portions, using only the first portion to generate a fingerprint for the search list. This segmentation approach maintains reliable redaction coverage while reducing the complexity of managing comprehensive search lists, as only partial fingerprints need to be stored and matched
Solution Approach 2:
The system applies different quality levels to different parts of the secret. The first portion is used for fingerprint generation and search list maintenance, while the second portion remains protected but is not stored in the search list. This local differentiation reduces overall system complexity while maintaining security
3Measurement precision
If the system splits secrets into multiple portions and uses entropy-based filtering to register only high-entropy portions for protection, then the precision of secret protection is improved, but the complexity of the protection mechanism increases
Solution Approach 1:
The system changes the parameter of protection by using entropy as a criterion to determine which portions of secrets to protect. By calculating entropy values and comparing them against thresholds, the system precisely identifies high-entropy portions that need protection, improving protection precision while using a relatively simple entropy calculation mechanism
Data Source
AI summary
A computer-implemented method for automatically redacting logs, comprising receiving a secret associated with a request for service, splitting the secret into a first portion and a second portion, determining whether the second portion has an entropy value greater than a predefined threshold, and in response to the entropy value being greater than the threshold, registering the first portion for protection by a logging service.