Pre-stored Connection Data for Real-time Job Notifications
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In online services with large user bases, such as LinkedIn, finding job postings from connections within a short time frame is computationally expensive due to the need to traverse extensive social graphs, leading to inefficiencies in resource utilization and response times.
Innovation Solution
A new data storage approach that pre-stores connection information for each node, allowing for quick retrieval of first- and second-degree connections, reducing the need for extensive database queries and enabling real-time job post notifications based on relevance to the member, using a proprietary derived-data key-value storage platform like Venice.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the system traverses extensive social graphs to find job postings from connections, then the completeness of job posting discovery is improved, but the computing resources and response time deteriorate
Solution Approach 1:
The system pre-computes and stores connection information (first-degree and second-degree connections) in a derived-data key-value storage platform before it is needed. When a job posting is detected, the system can immediately query the pre-stored connection data without performing expensive graph traversals in real-time, thus resolving the contradiction between complete job posting discovery and fast response time
2Loss of information
If the system traverses extensive social graphs to find job postings from connections, then the completeness of job posting discovery is improved, but the computing resources required deteriorate
Solution Approach 1:
Connection data is pre-computed and cached in a dedicated key-value storage platform (Venice) before being needed for job posting searches. This eliminates the need to repeatedly traverse the extensive social graph whenever a job posting query occurs, significantly reducing computing resource consumption while maintaining complete job posting discovery capability
Solution Approach 2:
The system creates a copy of the connection graph data in a simplified key-value format stored in Venice. Instead of querying the original complex social graph structure every time, the system queries this pre-created copy which contains only the necessary connection information, reducing the computational complexity and resource requirements
Data Source
AI summary
Methods, systems, and computer programs are presented for creating notifications for resources shared or posted by connections of a member. One method includes an operation for accessing databases to collect member information that includes first-degree connections and information about each first-degree connection. Further, a list of first-degree connections is stored in a data store with information about each first-degree connection. The method further includes operations for detecting a resource associated with a first member, and for accessing the data store to obtain network connections (first- and second-degree connections) of the first member. For each network connection of the first member, the method determines if the resource is relevant for the network connection based on the retrieved member information. Further, the method includes generating a notification about the resource for one or more of the network connections based on a relevance of the resource to the network connection.