Scalable Recommendation Engine Stateless Processor Scaling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing recommendation engines face inefficiencies due to the need for large memory mirroring and prolonged deployment times of new servers, leading to overcapacity and increased costs, as well as latency issues during peak traffic surges.

Innovation Solution

Implementing a scalable recommendation engine with stateless processors that do not retain transaction data, allowing for rapid deployment and deactivation based on processing bandwidth thresholds, decoupling from memory servers that store contextual data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If memory is mirrored across all recommendation servers, then data consistency is ensured, but deployment time increases significantly

Engineering Contradiction:

Improvedata consistencyVSAvoiddeployment time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent separates the recommendation logic (stateless processors) from the data storage (memory servers), allowing independent deployment. When a new recommendation server is added, it does not need to copy the entire memory, reducing deployment time while maintaining data consistency through centralized memory servers.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a memory server as an intermediary that all recommendation servers access. Instead of mirroring memory across servers, the memory server acts as a central intermediary that provides data to stateless processors, ensuring consistency without requiring full memory copies on each server.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Speed

If recommendation servers are deployed before capacity is needed, then latency is reduced, but operational costs increase due to overcapacity

Engineering Contradiction:

Improveresponse latencyVSAvoidoperational costs

Core Design Contradiction:

SpeedVSLoss of energy

Solution Approach 1:

The patent implements dynamic scaling of recommendation servers based on real-time traffic conditions. The system can rapidly spin up or down stateless processors in response to changing demand, allowing the system to maintain low latency during peak traffic without incurring the costs of permanently over-provisioning capacity.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system dynamically changes operational parameters (number of active processors, resource allocation) based on traffic conditions. This allows the system to optimize between latency and cost by adjusting capacity in real-time rather than maintaining fixed overcapacity.

Inventive Principle:

Principle #35Parameter changes

3Adaptability or versatility

If stateless processors are used without local memory, then scalability improves, but data access speed may be affected

Engineering Contradiction:

ImprovescalabilityVSAvoiddata access speed

Core Design Contradiction:

Adaptability or versatilityVSSpeed

Solution Approach 1:

The patent merges the advantages of stateless processors (scalability) with fast data access by implementing a distributed memory server architecture. The memory servers are positioned close to the stateless processors and use optimized data structures (sorted lists, Bloom filters) to enable rapid data retrieval without requiring local memory copies on each processor.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11164235B2Scalable systems and methods for generating and serving recommendations

Publication Date: 2021.11.02 SALESFORCE INC

AI summary

A scalable recommendation engine includes stateless processors in communication with at least one memory server that stores contextual data. A router is configured to direct a recommendation request to a first stateless processor, which is configured to generate a recommendation using contextual data from the memory server. A controller monitors the available processing bandwidth of the stateless processors and deploys an additional stateless processor if the available processing bandwidth is less than a minimum available processing bandwidth threshold. The controller can remove from deployment a stateless processor if the available processing bandwidth is greater than a maximum pre-determined available processing bandwidth threshold.