Dynamic Correlation Score Generation for Large Data Sets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for generating correlation scores between arbitrary objects become computationally expensive and reduce recommendation quality when dealing with large data sets, often requiring costly matrix maintenance and data set restrictions.
Innovation Solution
A network system with content servers, client computers, and a management server that generates seed objects, retrieves and filters lists of objects, computes interaction scores, and creates a list of correlated items using collaborative filtering techniques, allowing for dynamic generation of correlation scores.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If the data set is restricted by reducing the number of users or items to reduce computational cost, then the computational expense is reduced, but the quality of recommendations deteriorates
Solution Approach 1:
The patent segments the collaborative filtering computation into multiple passes: first identifying pivot items based on seed object interactions, then using those pivot items to generate candidate recommendations. This segmentation allows the system to process large data sets efficiently by breaking down the computational task into manageable stages rather than computing all-to-all correlations at once.
Solution Approach 2:
The patent performs preliminary action by pre-identifying pivot items that have interacted with the seed object before generating final recommendations. This preliminary filtering step reduces the search space for subsequent recommendation generation, allowing the system to maintain high recommendation quality while reducing computational expenses by focusing only on relevant items.
2Loss of information
If very large matrices are maintained for correlation calculations, then complete correlation data is available, but computational cost and memory requirements increase
Solution Approach 1:
The patent extracts only the necessary correlation data by identifying pivot items that have directly interacted with the seed object, rather than maintaining and processing complete correlation matrices for all items. This extraction approach retrieves only the relevant subset of correlation information needed for accurate recommendations, avoiding the computational burden of processing entire large matrices.
Solution Approach 2:
The patent implements dynamic generation of correlation scores by computing correlations on-demand based on seed object interactions rather than pre-computing and storing all possible correlations. This dynamic approach adapts to each user's specific interaction history, maintaining information completeness for relevant items while avoiding storage and computation of irrelevant correlation data.
3Adaptability or versatility
If arbitrary objects are correlated using collaborative filtering, then comprehensive recommendations can be generated, but computational complexity increases
Solution Approach 1:
The patent segments the correlation computation process into distinct phases: identifying items that interacted with the seed object (pivot items), then using those pivot items to generate candidate recommendations. This segmentation enables comprehensive correlation across arbitrary objects while managing computational complexity by processing relationships in manageable stages rather than computing all correlations simultaneously.
Solution Approach 2:
The patent introduces pivot items as intermediaries between the seed object and final recommendations. These pivot items serve as mediators that connect the user's interaction history to potential recommendations, enabling comprehensive correlation across arbitrary objects while simplifying the computational process by working through intermediate results rather than direct pairwise comparisons.
Data Source
AI summary
Systems and methods are described for performing the dynamic generation of correlation scores between arbitrary objects. When a behavioral event is recorded, that is to say when an end user interacts with multiple objects, relationships between objects are created. These relationships are maintained as a list. When a request for correlated items is requested based upon a seed object, a list of correlated items is dynamically created through the generation of a pivot set and a scoring algorithm to compute the list of correlated items.