Text Object Management System Using Density Estimation for Storage Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing data objects with textual content in databases is inefficient due to storage resource consumption, as irrelevant text objects accumulate and are often deleted, leading to potential loss of relevant data.
Innovation Solution
A system that generates a relevancy score for text objects using an unsupervised density estimation model, based on vectorized representations of text content, to determine whether text objects are relevant or irrelevant, allowing for targeted storage actions such as deletion or retention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If emails are kept indefinitely in the database, then data retention is improved, but storage resource consumption increases
Solution Approach 1:
The system changes the parameter of email management from binary (keep/delete) to continuous (relevancy scoring). By assigning relevancy scores based on multiple criteria (user profile, access patterns, content analysis), the system transforms storage decisions into a graded parameter system, allowing emails to be retained, archived, or deleted based on their scored relevancy rather than a simple time-based threshold.
Solution Approach 2:
The system enables self-service by automatically evaluating and scoring emails without requiring user intervention. The automated relevancy assessment system continuously monitors and scores emails based on user behavior patterns and content characteristics, allowing the database to self-manage its contents by identifying and retaining only high-value emails while automatically purging low-value ones.
2Quantity of substance
If emails are deleted after a fixed period, then storage resource consumption is reduced, but relevant data object loss occurs
Solution Approach 1:
The system performs preliminary action by proactively scoring and categorizing emails before deletion decisions are made. Rather than waiting for the retention period to expire and then deleting, the system continuously evaluates emails in advance, assigning relevancy scores and identifying which emails are likely to be needed, thereby preventing loss of relevant information before it occurs.
Solution Approach 2:
The system implements feedback loops where user interactions with emails (access patterns, reading behavior, forwarding) continuously update the relevancy scores. This feedback mechanism allows the system to learn from actual user behavior and adjust retention decisions dynamically, ensuring that emails which become relevant are retained even if they approached deletion thresholds.
3Reliability
If manual email management is performed by users, then relevant data retention is improved, but ease of operation deteriorates
Solution Approach 1:
The system replaces manual user management with automated self-service evaluation. The relevancy scoring system automatically assesses each email's importance based on user profiles, access patterns, and content analysis, eliminating the need for users to manually review and decide which emails to keep or delete, thereby maintaining high retention accuracy while preserving ease of operation.
Solution Approach 2:
The system introduces an intermediary layer (the relevancy scoring algorithm) between the user and the email database. This intermediary automatically performs the complex evaluation work that would otherwise require user attention, translating raw email data into scored recommendations that users can review or accept automatically, thus bridging the gap between automated efficiency and user control.
Data Source
AI summary
Systems and methods are disclosed for managing data objects that include text content that are stored in a database. The management of text objects includes determining that a condition for a data object associated with a user has been satisfied. In response, a relevancy score for the data object is generated. The relevancy score is based on the text content of the data object and a density estimation model associated with the user. The density estimation is generated using a plurality of data objects that each include text content and that are associated with a plurality of users of a service associated with the data objects, and using a set of the plurality of data objects that are associated with the user. Irrelevancy actions or relevancy actions may be performed to the data object based on the relevancy score.