Modular Recommendation Engine Testing Framework

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current personalization engines are rigid and require technical intervention for feature development, limiting their flexibility and scalability, making it difficult for large organizations to provide tailored content to a large customer base efficiently.

Innovation Solution

A modular architecture for a personalization engine with loosely-coupled modules for eligibility assessment, machine learning, and user-defined functionality, allowing non-technical personnel to define parameters and objectives, and utilizing machine learning and artificial intelligence to analyze customer metrics and optimize content delivery.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If personalization engines are developed with complex machine learning and deep learning components, then content personalization capability is improved, but device complexity and operational rigidity increase

Engineering Contradiction:
Improvecontent personalization capabilityVSAvoidarchitecture complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The personalization engine is divided into distinct modules: a machine learning module that handles complex algorithms, a content module that manages content delivery, and a testing module that evaluates performance. This segmentation allows each module to be developed and modified independently, reducing overall system complexity while maintaining advanced personalization capabilities.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The engine is designed with universal interfaces and standardized data structures that allow the same core architecture to handle multiple personalization scenarios and content types. This multi-functionality reduces the need for separate complex systems for different personalization tasks.

Inventive Principle:
Principle #6Universality (Multi-functionality)

2Device complexity

If personalization engines include preconfigured limitations such as predefined limits in content inputs, then device complexity is reduced, but adaptability and versatility deteriorate

Engineering Contradiction:
Improvesystem configuration simplicityVSAvoidfeature flexibility
Core Design Contradiction:
Device complexityVSAdaptability or versatility

Solution Approach 1:

The engine employs dynamic configuration parameters that can be adjusted at runtime without requiring system reconfiguration or technical intervention. Content input limits, algorithm selection, and testing parameters are all dynamically adjustable, allowing the system to adapt to different organizational needs while maintaining a simple base configuration.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The system allows non-technical users to modify key parameters such as content delivery thresholds, algorithm choice, and testing group sizes through a user-friendly interface. These parameter changes enable feature flexibility without increasing system complexity, as the underlying architecture remains unchanged.

Inventive Principle:
Principle #35Parameter changes

3Manufacturing precision

If new features or content require retooling and reconfiguration by engaging technology teams, then manufacturing precision is maintained, but productivity and loss of time increase

Engineering Contradiction:
Improvefeature implementation accuracyVSAvoidfeature development speed
Core Design Contradiction:
Manufacturing precisionVSProductivity

Solution Approach 1:

Non-technical personnel are empowered to add new content, modify personalization rules, and configure testing parameters directly through the engine's interface. This self-service capability eliminates the need to engage technology teams for routine feature additions, significantly improving productivity while maintaining implementation accuracy through standardized validation processes.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The engine includes pre-built templates, standardized content structures, and default algorithm configurations that allow non-technical users to quickly implement new features without extensive setup. These preliminary preparations maintain precision by ensuring all features adhere to established standards while accelerating deployment.

Inventive Principle:
Principle #10Preliminary action

4Adaptability or versatility

If personalization engines are designed to provide tailored content to large customer bases, then adaptability is improved, but device complexity and operational difficulty increase

Engineering Contradiction:
Improvecustomer base scalabilityVSAvoidoperational simplicity
Core Design Contradiction:
Adaptability or versatilityVSEase of operation

Solution Approach 1:

The engine introduces a testing module as an intermediary between the machine learning module and the content module. This testing module automatically evaluates different personalization strategies on subsets of the customer base before full deployment, simplifying operations by providing data-driven guidance on which personalized content configurations will succeed at scale without requiring complex manual intervention.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS12045298B1Testing framework to measure impact of variant within large scale recommendation engine
Publication Date: 2024.07.23 MORGAN STANLEY SERVICES GROUP INC
  • US12045298B1 patent drawing
  • US12045298B1 patent drawing
  • US12045298B1 patent drawing

AI summary

At least one processor is configured for defining a plurality of mutually exclusive customer treatment groups, including in accordance with first and second algorithms, to receive content items. Content items are respectively provided to a random customer treatment group as well as first and second algorithm customer treatment groups, and metrics representing at least engagement by each of the customers are determined and analyzed. A selection of the first or the second algorithm is made. The at least one processor is configured to provide, to at least some of the plurality of customers, content items in accordance with the selected algorithm.