User Experience Evaluation Using Behavior Logs and POMDP

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional techniques for customizing interactive computing environments based on consumer experiences are limited by low response rates from user surveys and the reliability of survey-based feedback, which are biased and difficult to correlate with specific interactions.

Innovation Solution

The use of user experience evaluation techniques that collect and analyze interaction data from behavior logs to determine user experience values, employing Partially Observable Markov Decision Process (POMDP) and model-based or model-free approaches to assess user interactions and provide actionable insights for improving user experiences without requiring user feedback.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If survey-based feedback is used to measure consumer experience, then user experience data can be collected, but response rates are very low and reliability is poor

Engineering Contradiction:

Improveuser experience measurement reliabilityVSAvoidfeedback collection efficiency

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system automatically collects interaction data from behavior logs without requiring user participation in surveys. The computing device itself generates the feedback data through unobtrusive tracking of user interactions, eliminating the need for users to manually respond to survey requests.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces the mechanical survey system (requiring user action to complete questionnaires) with an automated data collection system that passively monitors and records interaction events from behavior logs, transforming the feedback mechanism from active user participation to passive system observation.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Loss of information

If survey responses are collected, then some user experience information is obtained, but it is difficult to correlate responses with specific interactions

Engineering Contradiction:

Improvecontext information retentionVSAvoiddata correlation complexity

Core Design Contradiction:

Loss of informationVSDevice complexity

Solution Approach 1:

The system pre-establishes a detailed record of interaction events in behavior logs before survey responses are needed. Each interaction is timestamped and stored with contextual information, creating a ready-to-analyze dataset that can be directly correlated with user experience metrics without requiring post-hoc matching.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent creates a digital copy of the interaction sequence from behavior logs that parallels the survey response timeline. This copied interaction data serves as a reference framework that makes it straightforward to map survey responses to specific interaction events, eliminating the need for complex correlation algorithms.

Inventive Principle:

Principle #26Copying

3Reliability

If survey questions are asked to users, then user experience feedback is obtained, but the feedback is biased and provides only a snapshot rather than comprehensive view

Engineering Contradiction:

Improveuser experience data accuracyVSAvoidcomprehensiveness of experience data

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The system continuously collects interaction data throughout the entire user session rather than taking a single snapshot at the end. Every interaction event is recorded in real-time, providing a continuous stream of behavioral data that captures the complete user experience journey from start to finish.

Inventive Principle:

Principle #20Continuity of useful action

Solution Approach 2:

The patent collects excessive detail about user interactions, recording every click, hover, and navigation event beyond what traditional surveys capture. This excessive data collection ensures that no relevant experience information is missed, and the comprehensive dataset can be selectively analyzed to derive accurate experience metrics.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11551239B2Characterizing and modifying user experience of computing environments based on behavior logs

Publication Date: 2023.01.10 ADOBE INC

AI summary

There is described a method and system in an interactive computing environment modified with user experience values based on behavior logs. An experience valuation system determines an experience value and an estimated experience value. The experience value is based on a current state of interaction data from a user session, based on a history of past events, and an estimation function defined by parameters to model the user experience values. The estimated experience value is determined based on, in addition to the current state and the estimation function, next states associated with the current state, and a reward function. The parameters of the estimation function are updated based on a comparison of the expected experience value and the estimated experience value. For another aspect, the method and system may further include a state prediction system to determine probabilities of transitioning that may be applied to determine the estimated experience value.