Item Group Lift Scoring for Online Concierge Feature Testing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional feature testing methodologies for online concierge systems are ineffective in quantifying how new features impact user interactions with items presented as part of item groups, inhibiting the ability to improve item presentation strategies.

Innovation Solution

An online concierge system generates an aggregated lift score by tracking user interactions with item groups, computing interaction metrics for both test and control sets, and calculating group lift scores to evaluate the performance of new features across multiple item groups, thereby determining their impact on user interactions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional feature testing methodologies are used to test new features, then individual item interaction improvements can be quantified, but the overall impact on user interactions with item groups cannot be measured

Engineering Contradiction:

Improvemeasurement capabilityVSAvoidapplicability to item groups

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent combines multiple individual item interaction metrics into a single aggregated lift score that measures the overall impact on item group interactions. This merging approach allows the system to evaluate new features at the item group level while maintaining the precision of individual item measurements.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent introduces a new dimension of measurement by creating an aggregated metric that operates at the item group level rather than just the individual item level. This dimensional change enables comprehensive evaluation of feature impact across entire item groups while preserving granular measurement capabilities.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Reliability

If new features are tested across all users, then comprehensive performance data can be collected, but computational resources are excessively consumed

Engineering Contradiction:

Improveperformance evaluation accuracyVSAvoidcomputational resource consumption

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The patent implements partial testing by evaluating new features on a subset of users rather than all users. This partial action approach maintains sufficient statistical reliability for performance evaluation while significantly reducing computational resource consumption compared to full-scale deployment.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The patent performs preliminary feature testing and evaluation before full deployment. By conducting initial assessments on subsets of users and using aggregated lift scores to predict overall performance, the system可以避免 the need to process data from all users, thereby reducing computational energy consumption while maintaining evaluation reliability.

Inventive Principle:

Principle #10Preliminary action

3Productivity

If prioritized items are presented to all users, then item provider goals are maximized, but user interaction quality decreases due to reduced personalization

Engineering Contradiction:

Improveitem provider objective achievementVSAvoiduser interaction quality

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The patent implements dynamic item group presentation by using aggregated lift scores to determine which item groups receive prioritized presentation for different users. This dynamic approach allows the system to adapt prioritization strategies based on user characteristics and interaction patterns, thereby maintaining user interaction quality while still achieving item provider objectives.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent applies prioritized item presentation selectively to specific item groups rather than uniformly to all items. By using aggregated lift scores to identify which item groups benefit most from prioritization, the system maintains high user interaction quality for personalized recommendations while still advancing item provider goals for specific item groups.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS12536576B2Method, medium, and system for scoring improvements by test features to user interactions with item groups

Publication Date: 2026.01.27 MAPLEBEAR INC

AI summary

An online concierge system generates an aggregated lift score for a test feature for the online concierge system. The online concierge presents prioritized items from a set of item groups to two sets of users: a test set and a control set. The online concierge system uses the test feature to present prioritized items to users in the test set, and the online concierge system uses existing functionality to present prioritized items to users in the control set. For each test group, the online concierge system creates holdout subsets out of the test set and the control set. The online concierge system tracks user interactions with items in an item group and computes a group lift score for the item group. The online concierge system generates an aggregated lift score for the test feature based on the group lift scores and presents items to users based on the aggregated lift score.