Database Clustering for Search Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
E-commerce database systems face performance issues due to increased latency as the number of products grows, leading to slow search engine performance, and existing strategies for indexing products are inefficient, resulting in high false negative rates when predicting product access likelihood.
Innovation Solution
Implementing a prioritized product index structure using a supervised machine learning model that classifies products as 'hot' or 'cold' based on historical access data, with a cost-sensitive algorithm to minimize errors, and incorporating features like seasonality and newness to improve prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the number of products in the database system increases, then the variety of products available for purchase increases, but the performance of the database system slows down due to increased latency
Solution Approach 1:
The patent divides the database system into multiple database clusters (first database cluster and second database cluster) to segment the large dataset. This segmentation allows the system to handle increased product quantities by distributing data across clusters, thereby maintaining search performance and reducing latency when dealing with large numbers of products.
Solution Approach 2:
The patent implements different database clusters with potentially different configurations and performance characteristics. By assigning specific products to specific clusters based on access patterns and characteristics, the system optimizes local query performance while handling overall system growth, thus managing the trade-off between product quantity and search latency.
2Device complexity
If traditional indexing strategies are used to organize products, then the database structure remains simple, but the false negative rate in predicting product access likelihood increases
Solution Approach 1:
The patent employs a supervised machine learning model that uses feedback from historical access data to continuously improve prediction accuracy. The model learns from past product access patterns and adjusts its predictions accordingly, reducing false negatives while maintaining a manageable indexing structure through intelligent rather than purely structural complexity.
Solution Approach 2:
The patent replaces traditional mechanical indexing strategies with a machine learning-based predictive system. Instead of relying solely on structural organization methods, the system uses computational models that analyze historical data to predict product access likelihood, thereby improving reliability without proportionally increasing structural complexity.
Data Source
AI summary
Various embodiments can include a system that can comprise: one or more processing modules and one or more non-transitory memory storage modules storing computing instructions configured to run on the one or more processing modules. The computing instructions can be configured to perform acts of: accessing a query log, wherein the query log can comprise click-through information and item-view information for a record in a set of distinct records in a database system, wherein the database system can comprise a first database cluster H on a first database server and a second database cluster L on a second database server, wherein: the first database server can have a greater processing power than the second database server; the click-through information can comprise information regarding how many times the record was clicked by any user; and the item-view information can comprise information regarding how many times the record was transmitted to be displayed on any display device of any user; calculating a total click-through for the record by determining how many times the record was clicked by any user during a time period; calculating a total item-view for the record by determining how many times the record was transmitted to be displayed on any display device of any user during the time period; storing the record in either the first database cluster H or the second database cluster L based on the total click-through for the record and the total item-view for the record; receiving a search request from a requester; and transmitting a search result to be displayed on a display device of the requester, the search result comprising the record. Other embodiments are described herein.