Time-scale-aware modulation system for generating sign language.

Abstract

The invention relates to a time-scale awareness modulation system for generating sign language, which includes a motion space module, a condition space module, and a diffusion space module for sign language generation. By using two parallel processing text encoding sets, CLIP and BERT, to process the text sequence before embedding it with a time-embedding vector to generate a condition-embedding vector, and simultaneously diffusing the posture sequence over time t, the combined processing allows simultaneous downsampling and upsampling on both spacetime dimensions and the collection of motion information across multiple time scales, enabling accurate sign language generation, preserving time details, and reducing computational resources.

Images