Physical Document Content Replacement Using OCR and Affinity Scoring

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional documents, both physical and digital, lack customization for individual users, leading to irrelevant content that reduces the likelihood of customers obtaining items and inefficiently uses display space on client devices.

Innovation Solution

An online system personalizes content by extracting components from physical documents using OCR and VLMs, applying interaction models to predict user interactions, and modifying or replacing content based on user characteristics to generate tailored alternative documents.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If static content is preselected for documents to simplify creation, then document creation is simplified, but the document cannot include content customized for individual customers

Engineering Contradiction:

Improvedocument creationVSAvoidcontent customization

Core Design Contradiction:

Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The document content is segmented into multiple components (item images, item descriptions, promotional content) that can be individually processed and selected. The system extracts these components from source documents and selectively includes only relevant components for each user based on their characteristics, enabling customization while maintaining efficient processing.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system changes the parameter of content selection from static preselection to dynamic selection based on user characteristics. By analyzing user characteristics (preferences, behavior patterns, demographics) and adjusting which content components are included, the system achieves personalized documents while maintaining automated processing.

Inventive Principle:

Principle #35Parameter changes

2Stability of the object's composition

If static content is included in documents to maintain consistency, then content consistency is maintained, but irrelevant content reduces the likelihood of customers obtaining items

Engineering Contradiction:

Improvecontent consistencyVSAvoidcustomer item acquisition

Core Design Contradiction:

Stability of the object's compositionVSReliability

Solution Approach 1:

The system applies local quality by tailoring specific content components to individual users based on their characteristics. Different users receive different sets of content components (items, descriptions, promotions) within the document, optimizing relevance for each user while maintaining the overall document structure and consistency.

Inventive Principle:

Principle #3Local quality

3Ease of operation

If static content is presented to all users to simplify distribution, then distribution simplicity is maintained, but display area on client devices is inefficiently used

Engineering Contradiction:

Improvedistribution simplicityVSAvoiddisplay area utilization

Core Design Contradiction:

Ease of operationVSArea of stationary object

Solution Approach 1:

The system applies partial action by selecting and presenting only the necessary content components for each user based on their characteristics and interests. Instead of presenting all static content to every user, the system includes only relevant components, optimizing display area utilization while maintaining simplified automated distribution.

Inventive Principle:

Principle #16Partial or excessive action

4Device complexity

If items are preselected for documents to reduce complexity, then document complexity is reduced, but the document cannot account for user-specific preferences

Engineering Contradiction:

Improvedocument complexityVSAvoiduser preference adaptation

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The system enables self-service by automatically analyzing user characteristics and selectively including relevant content components without manual intervention. The automated process extracts, evaluates, and selects content components based on user profiles, achieving personalized documents while maintaining low operational complexity.

Inventive Principle:

Principle #25Self-service

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

The personalized documents increase the likelihood of user interactions by including relevant content, optimizing display space on client devices, and enhancing the influence of promotional content on users.

Implementation Method 1

The online system applies a combination of models to an image of the physical document that extracts different components of the physical document. The online system applies a combination of one or more optical character recognition (OCR) models

Methodology Applied

Scientific EffectOptical character recognition:

Implementation Method 2

The online system applies a combination of one or more visual language models (VLMs) to the image of the physical document to extract components of the physical document based on text and images included in the physical document

Methodology Applied

Scientific EffectVisual language modeling:

Data Source

PatentUS12536183B1Identifying and modifying components of a physical document using machine-learning models

Publication Date: 2026.01.27 MAPLEBEAR INC

AI summary

An online system customizes documents for a particular context, user, or set of users. The online system receives an image of a physical document and extracts components, such as text, titles, items and their metadata, from the physical document. The online system may apply rules to the metadata for one or more items to determine whether to modify at least a portion of the metadata. The online system also applies a model to generate an affinity score for a context or a user and each component of the document. If the score for a component is below a threshold, the online system prompts a generative model to generate replacement content for the component. Subsequently, the online system applies the model to the generated replacement content and updates the document with the generated replacement content for the component if the score of the generated replacement content is higher.