Spatio-Temporal Sign Pose Generation for Natural Multi-Channel Signing

The patent segments the sign language generation task into multiple independent channels corresponding to different body parts (hands, arms, torso, head, face). Each channel processes spatial and temporal features separately through dedicated transformer encoders and decoders, then combines them to generate comprehensive sign pose sequences. This segmentation enables accurate modeling of complex multi-channel movements while maintaining modular architecture.

The patent introduces spatio-temporal dimensionality by processing sign pose data across multiple spatial channels (different body parts) and temporal dimensions (sequence of poses over time). The transformer architecture captures spatial relationships between body parts and temporal dependencies in motion sequences, transforming the simple text-to-pose mapping into a multi-dimensional generation task that preserves natural sign language expressiveness.

The patent employs a universal transformer architecture that handles multiple functions: text encoding, spatial feature extraction for different body parts, temporal sequence modeling, and pose generation. The same transformer block structure is reused across different channels and processing stages, reducing overall complexity despite the multi-channel design. This multi-functional approach allows a single model to capture diverse aspects of sign language production.

The patent implements a nested architecture where temporal transformer encoders and decoders are embedded within the broader spatio-temporal processing framework. The temporal attention mechanisms are nested within the spatial channel processing, creating a hierarchical structure where smaller functional units (temporal processing) are contained within larger ones (spatio-temporal processing). This nesting enables efficient feature extraction at multiple levels without linearly increasing complexity.

The patent applies preliminary action by pre-processing the input text sequence through a text encoder to extract meaningful contextual features before feeding them into the multi-channel spatio-temporal processing pipeline. This preliminary text encoding prepares the semantic information in advance, enabling the subsequent attention mechanisms to focus on spatial and temporal patterns without redundant computation, thereby improving processing efficiency while maintaining high expressiveness.

The patent introduces contextual features as an intermediary between the input text and the spatio-temporal processing stages. The text encoder generates contextual representations that mediate the transformation from textual semantics to visual pose sequences. This intermediary layer bridges the gap between language understanding and motion generation, enabling more accurate and expressive sign language synthesis while organizing the computational flow into manageable stages.