Item Quality Assessment Using Multimodal Feedback for Pickers

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current online concierge systems struggle to scale quality assessment of picked items due to the complexity of managing multiple user preferences, leading to inefficiencies in ensuring high-quality item selection by pickers.

Innovation Solution

Utilizing a multi-modal language model to automatically assess item quality by analyzing images and user preferences, generating feedback on potential issues, and providing real-time suggestions to pickers.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the online concierge system manually provides suggestions to pickers about item quality, then pickers can consider user preferences, but the system cannot scale to handle multiple preferences efficiently

Engineering Contradiction:

Improveability to consider multiple user preferencesVSAvoidscaling capability of quality assessment

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent replaces the manual mechanical process of pickers evaluating items with an automated computer vision system using machine learning models. The system captures images of items, processes them through trained models to assess quality attributes, and provides feedback automatically, eliminating the need for manual evaluation while handling multiple user preferences simultaneously.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system enables self-service quality assessment where the computer vision model autonomously evaluates items without human intervention. The model independently analyzes images, compares items against quality standards and user preferences, and generates feedback, making the system self-sufficient and scalable.

Inventive Principle:

Principle #25Self-service

2Manufacturing precision

If the system provides detailed quality guidance to pickers, then item quality improves, but the complexity of managing multiple preferences increases

Engineering Contradiction:

Improvequality of picked itemsVSAvoidcomplexity of preference management

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent introduces a computer vision system as an intermediary between user preferences and picker actions. The system translates complex multiple preferences into simplified visual feedback about item quality, acting as a mediator that handles the complexity internally while presenting simple, actionable guidance to pickers.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system implements a feedback loop where the computer vision model analyzes item images, compares them against quality standards and user preferences, and provides real-time feedback to pickers. This feedback mechanism guides pickers to select higher quality items while the system internally manages the complexity of multiple preferences through the machine learning model.

Inventive Principle:

Principle #23Feedback

3Measurement precision

If pickers manually evaluate items against multiple user preferences, then quality assessment is thorough, but the picking process becomes time-consuming

Engineering Contradiction:

Improveaccuracy of quality assessmentVSAvoidtime for picking process

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent replaces the time-consuming manual evaluation process with automated computer vision technology. The system captures images of items and uses trained machine learning models to rapidly assess quality attributes, providing accurate evaluations in seconds rather than requiring pickers to manually inspect and compare items against multiple preferences.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system performs preliminary quality assessment by analyzing item images before pickers make selection decisions. The computer vision model pre-evaluates items against quality standards and user preferences, providing advance feedback that guides picker choices and eliminates the need for time-consuming manual verification during the picking process.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12547649B2Automatic quality assessment of an item during order fulfillment

Publication Date: 2026.02.10 MAPLEBEAR INC

AI summary

Use of a language model to automatically perform visual assessment of quality of an item being fulfilled by a picker. The online system receives an image of the item and identifies a set of potential problems associated with the item. The online system generates a plurality of prompts for input into the language model including the image and one or more questions each corresponding to a respective potential problem of the set potential problems. The online system requests the language model to generate, based on the plurality of prompts, a feedback response for each potential problem. The online system generates an aggregated output by aggregating the feedback response for each potential problem, and based on the aggregated output, a second message that identifies one or more relevant problems associated with the item. The online system causes a device of the picker to display the second message.