Off-Policy Evaluation via Action Embedding Vectors

The patent introduces embedding vectors as an intermediary representation between the logging policy and target policy. These embedding vectors map actions and contexts into a continuous vector space, enabling the evaluation system to handle unseen actions in the target policy by finding similar actions in the embedding space. This intermediary representation resolves the contradiction by maintaining estimation accuracy while increasing flexibility to evaluate diverse target policies.

The patent transforms the discrete action space into a continuous embedding space by changing the parameter representation of actions. Instead of working directly with discrete action identifiers, the system uses continuous vector representations that capture semantic similarities. This parameter transformation enables accurate performance estimation for target policies that include unseen actions, as the embedding space allows for generalization to novel actions.

The patent creates compressed copies of the action space through embedding vectors. Instead of processing the entire large action space directly, the system uses a lower-dimensional embedding representation that captures the essential structure and relationships. This copying approach maintains the ability to handle large action spaces while dramatically improving computational efficiency by operating in the compressed embedding space.

The patent transforms the high-dimensional discrete action space into a lower-dimensional continuous embedding space. By projecting actions into a reduced-dimensional vector space that preserves semantic relationships, the system can efficiently process large action spaces without sacrificing the ability to evaluate comprehensive policies. This dimensionality reduction resolves the contradiction between handling large action spaces and maintaining computational efficiency.