An
augmented reality system in which video imagery of a physical environment is combined with video images output by a
game engine by the use of a traveling matte which identifies portions of the visible physical environment by techniques such as
Computer vision or chroma
keying and replaces them with the video images output by the
video game engine. The composited imagery of the physical environment and the
video game imagery is supplied to a trainee through a headmounted display screen. Additionally,
peripheral vision is preserved either by providing complete binocular display to the limits of
peripheral vision, or by providing a visual path to the
peripheral vision which is matched in luminance to higher resolution
augmented reality images provided by the binocular displays. A
software / hardware element comprised of a
server control
station and a controller onboard the trainee performs the modeling,
scenario generation, communications, tracking, and metric generation.