VR training system with token-based control of a synthetic audience using an affective resonance field

Abstract

VR training system with synthetic audience for training communication skills, comprehensive a. a recording unit for capturing multimodal input from a presenter, b. an encoder that generates a sequence of discrete interaction tokens from the multimodal inputs, c. an agent engine for managing the affective states of a number of synthetic audience agents, d. a resonance field engine for calculating a population-wide affective resonance field depending on affective states, e.g. a reaction generation that derives reaction tokens for the audience agents from the affective states and the resonance field, and f. a rendering unit for displaying the reaction tokens in a VR environment, wherein the agent engine, the resonance field engine and the reaction generation are set up for synchronous updating per interaction token.