According to the present invention, even if a plurality of operators perform gesture operations in three-dimensional space, the three-dimensional spatial gesture of each operator is accurately associated with the target object of the three-dimensional spatial gesture. For this, a projector displays at least one selection-target object on an upper surface of a table. When a pointer operated by an operator comes in contact with the upper surface of the table, a two-dimensional coordinate detecting apparatus detects the contact position, and determines which object has been designated by the operator. In this stage, among pointers detected by a three-dimensional coordinate detecting apparatus, a pointer having a position closest to the contact position is determined, and the pointer determined to be the closest is decided as a tracking target. Thereafter, the tracking-target pointer is tracked by the three-dimensional coordinate detecting apparatus to determine the operator's gesture pattern.