Real-Time Search Index Update via Event Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current solutions for updating online profiles and menu information on food ordering websites process requests in batches, resulting in significant time delays and inefficiencies, causing inconvenience to users as changes are not reflected in real time.
Innovation Solution
A system and method that involve receiving entity change requests, saving them to an entity store, pushing them to an event publisher for forwarding to a streaming service, classifying the requests, and updating a search index database in real time to ensure immediate propagation of updates across user devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If batch processing is used for updating online profiles and menu information, then system complexity is reduced and processing is simplified, but significant time delays occur and updates are not reflected in real time
Solution Approach 1:
The patent segments the update processing into multiple independent components: event publishers that generate update events, streaming services that transport events, classification modules that categorize events by entity type, and search index databases that store and serve updated data. This segmentation enables real-time processing while keeping each component relatively simple and modular.
Solution Approach 2:
The patent introduces several intermediary components between the update source and the final search index: event publishers act as intermediaries to capture and standardize update events, streaming services serve as intermediaries to transport events in real-time, and classification modules act as intermediaries to route events to appropriate search indexes. These intermediaries enable real-time processing without requiring complex direct connections between all system components.
2Productivity
If real-time processing is implemented for entity change requests, then update speed and user experience improve, but system complexity and infrastructure requirements increase
Solution Approach 1:
The patent creates universal components that handle multiple types of entity updates through a single standardized interface. The event publisher framework can handle updates for merchants, menus, items, and other entities uniformly. The streaming service and classification module provide universal processing pathways that work for all entity types, reducing the need for separate specialized systems for each update type.
Solution Approach 2:
The patent implements preliminary action by pre-establishing the event publishing infrastructure, streaming services, and search index databases before updates are needed. Event publishers are pre-configured to recognize and publish various entity change events, and the classification system is pre-set up with knowledge of different entity types and their corresponding search indexes. This preliminary setup enables immediate real-time processing when updates occur without requiring complex runtime decision-making.
3Reliability
If all entity change requests are propagated to all search indexes, then data consistency is maintained, but data overhead and processing inefficiency increase
Solution Approach 1:
The patent applies local quality by making the data propagation selective rather than universal. The classification module examines each event and determines which specific search indexes need to be updated based on the event type and entity involved. Only the relevant local search indexes receive and process the update events, while other indexes remain unchanged. This selective approach maintains data consistency for affected indexes while avoiding unnecessary processing overhead for unrelated indexes.
Data Source
AI summary
A system for receiving and propagating efficient search updates includes one or more processors configured to receive, from a first external system via a network, a first entity change request to modify data in an entity associated with the first external system. The first entity change request is saved in an entity store. The received entity change request is pushed from the entity store to an event publisher for forwarding to a streaming service. The first entity change request is classified and forwarded, from the streaming service, to a search index database. The search index is then updated based on the classified entity change request.