Contextual Thompson Sampling with Corrupted Context
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing contextual bandit methods fail to effectively address situations where the context information is corrupted or missing, leading to suboptimal decision-making in applications like online advertising and medical treatments.
Innovation Solution
A method combining contextual and classical multi-armed bandit approaches, using Thompson Sampling with corrupted and missing context (TSCMC), which selects actions based on a weighted linear combination of contextual and classical bandit functions, even when true context is not available, to maximize rewards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If standard contextual bandit methods are used, then decision-making can be performed with available context information, but performance deteriorates when context information is corrupted or missing
Solution Approach 1:
The patent combines contextual bandit and classical bandit approaches into a unified framework. When context information is available and reliable, the system uses contextual bandit methods; when context is corrupted or missing, it automatically falls back to classical bandit methods that do not depend on context information. This merging allows the system to maintain reliable decision-making across both scenarios.
Solution Approach 2:
The system dynamically adapts its approach based on the quality and availability of context information. It monitors whether context is corrupted or missing and switches between contextual and classical bandit strategies accordingly. This dynamic adaptation ensures that the decision-making process remains reliable regardless of context information quality.
2Productivity
If contextual bandit methods are used, then rewards can be maximized when context is accurate, but learning efficiency decreases when context is corrupted
Solution Approach 1:
The system uses partial contextual information when available but does not fully depend on it. By combining contextual bandit (which uses context) with classical bandit (which ignores context), the system can partially utilize context information when it's reliable while having a backup mechanism that doesn't suffer from corrupted context, thus maintaining learning efficiency.
Solution Approach 2:
The unified framework acts as an intermediary between contextual and classical bandit approaches. It mediates the decision-making process by determining which approach to use based on context quality, allowing the system to learn effectively from all iterations regardless of context corruption, thus reducing overall learning time.
3Adaptability or versatility
If a single bandit approach is used, then the system remains simple, but it cannot handle both corrupted and clean context scenarios effectively
Solution Approach 1:
The patent creates a universal bandit framework that performs multiple functions: it can operate as a contextual bandit when context is reliable and as a classical bandit when context is corrupted. This multi-functionality allows the system to handle both clean and corrupted context scenarios effectively without requiring entirely separate systems for each scenario.
Solution Approach 2:
By merging contextual and classical bandit approaches into a single unified framework, the system achieves versatility in handling different context scenarios while avoiding the complexity of maintaining completely separate systems. The unified framework integrates both approaches and automatically selects the appropriate strategy based on context quality.
Data Source
AI summary
A method, computer program product, and computer system for triggering actions within a multi-armed bandit process with corrupted context. In a current time step: a context vector c(t) is received; a weight a is sampled, from first normal probability distribution, to maximize a function fα of c(t) and ũa; functions f1k are f2k respectively having and not having a functional dependence on c(t) are determined for each arm k; an arm k(t) is selected to maximize [α(t)f1k+(1−α(t))f2k]; an electromagnetic signal is sent to a hardware machine directing the hardware machine to perform an action of the selected arm k(t); a dynamic reward rk(t) resulting from the hardware machine having performed the action is received; and updates are performed for the next time step, including updating the first normal probability distribution for α=α(t) as a function of c(t) and rk(t) for the selected arm k(t).