File Ingestion Estimation in Multi-Tenant Cloud
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In a multi-tenant cloud environment, estimating the ingestion time of ingested files into a searchable state is challenging due to unknown user data ingestion patterns and varying hardware resource usage by other users, making traditional on-premises estimation methods infeasible.
Innovation Solution
A method that divides ingested files into independent elements, calculates ingestion time based on the number of divisions, file size, and maximum simultaneous users, and stores this time as internal data within the indexed file, allowing for accurate estimation and filtering of search results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If traditional on-premises estimation methods are used in a multi-tenant cloud environment, then the estimation process is simple, but the estimation accuracy deteriorates due to unknown user data ingestion patterns and varying hardware resource usage
Solution Approach 1:
The patent segments the file into multiple divisions and processes each division independently to calculate ingestion time. This segmentation allows the system to account for varying resource usage patterns in a multi-tenant cloud environment by breaking down the overall ingestion process into manageable units that can be estimated more accurately.
Solution Approach 2:
The patent performs preliminary actions by determining the maximum number of simultaneous users before calculating ingestion time estimates. This preliminary determination of resource capacity and user patterns enables more accurate estimation without requiring complex real-time monitoring during the actual ingestion process.
2Device complexity
If file ingestion is processed as a single unit, then the processing is simpler, but the ingestion time estimation is less accurate due to variations in file size and divisibility
Solution Approach 1:
The patent divides the file into multiple divisions and calculates ingestion time for each division separately. This segmentation approach improves estimation precision by accounting for variations in file characteristics across different portions of the data, rather than treating the entire file as a uniform unit.
Solution Approach 2:
The patent changes parameters by considering both the number of divisions and the size of each division when calculating ingestion time. This multi-parameter approach allows for more precise estimation that adapts to the actual structure and characteristics of the ingested file.
3Measurement precision
If the system accounts for maximum simultaneous users and file divisions, then the ingestion time estimation becomes more accurate, but the calculation complexity increases
Solution Approach 1:
The patent performs preliminary determination of the maximum number of simultaneous users and file division characteristics before the actual ingestion time calculation. This preliminary action separates the complex determination steps from the final calculation, making the overall process more manageable while maintaining accuracy.
Solution Approach 2:
The system uses self-service by automatically determining file divisibility and calculating ingestion time based on inherent file properties and system capacity. This automated approach reduces manual intervention complexity while maintaining high estimation accuracy through systematic calculation methods.
Data Source
AI summary
Systems, methods, and computer programming products for estimating ingestion time of ingested files to be transformed into a searchable state for content mining by an on-premises computing environment or cloud environment, including multi-tenant cloud environments. Ingested files being indexed are analyzed for divisibility. Ingestion time varies based on the number of divisible elements (such as lines) of data within the ingested file and the amount of data per divisible element. A converter divides files into a plurality of elements treated as independent data and calculates the estimated ingestion time based on the number of divisions and file size for each divisible element. Estimated ingestion time is stored to internal fields corresponding to each divisible element in the index for the search data. During content mining, an internal condition is added to received search queries, displaying only search results where the estimated ingestion time is older than the current time.