SCouT Transformer for Synthetic Counterfactuals

The patent segments the intervention estimation problem into distinct spatiotemporal components processed by specialized transformer modules. The encoder-decoder architecture divides data processing into encoding (pre-intervention) and decoding (post-intervention) phases, with separate attention mechanisms for spatial and temporal relationships. This segmentation allows the complex model to handle noisy data systematically through modular processing stages.

The patent introduces explicit spatial and temporal embedding dimensions to transform the input data representation. By adding spatial embeddings (for donor-unit relationships) and temporal embeddings (for pre/post-intervention timing), the model processes data in an expanded dimensional space that captures spatiotemporal dependencies. This dimensional expansion enables reliable prediction by incorporating previously unmodeled relationship structures.

The patent creates a universal transformer-based framework that handles multiple data types and scenarios through a single architecture. The model processes continuous outcomes, discrete classes, and rare disease data using the same encoder-decoder structure with configurable embeddings. This universality achieves high adaptability across different disease types and outcome variables without requiring separate specialized models.

The patent achieves adaptability to rare diseases and discrete classes by dynamically adjusting model parameters including embedding dimensions, attention heads, and layer configurations. The spatial and temporal embeddings can be customized based on data characteristics, allowing the same base architecture to adapt to diverse scenarios from common to rare diseases through parameter modification rather than structural redesign.

The patent introduces latent representation vectors as intermediary structures between the raw noisy input data and the final predictions. The encoder transforms observed noisy outcomes into compressed latent representations that filter noise while preserving essential signal patterns. These intermediate latent variables act as mediators that reconcile the conflict between noise filtering and information preservation by selectively encoding relevant features.

The patent replaces traditional mechanical noise filtering methods (such as smoothing or thresholding) with a learned neural network-based filtering mechanism. Instead of applying fixed mathematical filters that may remove important signals, the transformer model learns adaptive filtering through training, substituting rigid mechanical filtering with flexible learned representations that preserve information while removing noise.