Gaze-Controlled MIDI Controller for Accessible Music Interfaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is no existing solution for users with limited body movement, such as ALS patients, to easily control musical sound generation devices.
Innovation Solution
A controller that acquires gaze information from the user, dynamically displays operating portions associated with control commands on a display unit, and uses this information to select and transmit MIDI standard commands to a musical sound generation device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional performance interfaces are used, then musical sound generation devices can be controlled, but users with limited body movement cannot easily operate them
Solution Approach 1:
The patent replaces mechanical operation systems (physical buttons, switches, and manual controls) with an eye-tracking based selection system. The gaze detection unit detects the user's line of sight to determine which operating portion should be selected, eliminating the need for physical movement to operate the musical sound generation device. This allows users with limited body movement to control the device simply by moving their eyes.
2Ease of operation
If gaze information is used for selection, then users with limited movement can control the device, but the interface complexity increases
Solution Approach 1:
The patent introduces a pointer as an intermediary element that visually indicates which operating portion will be selected based on gaze detection. The pointer is displayed on the display unit and moves according to the detected line of sight, providing a clear visual feedback mechanism that simplifies the interaction between the user's gaze and the operating portions. This intermediary pointer makes the selection process more intuitive and reduces the perceived complexity of the interface.
3Adaptability or versatility
If operating portions are dynamically displayed, then the interface becomes more adaptable, but the system complexity increases
Solution Approach 1:
The patent implements dynamic display of operating portions where the displayed elements can change position, appearance, or availability based on the detected gaze direction and selection state. The display control unit updates the visual presentation of operating portions in real-time as the user moves their eyes, creating an adaptive interface that responds to user intent without requiring complex reconfiguration of the underlying system architecture.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A controller that controls a musical sound generation device has an acquisition unit that acquires gaze information of a user, a display control unit that dynamically displays a plurality of operating portions associated with control commands for the musical sound generation device on a display unit, and displays a pointer for selecting an operating portion on the display unit on the basis of the gaze information acquired by the acquisition unit, a determination unit that determines selection of an operating portion overlapping at least a part of the pointer among the plurality of operating portions, and a communication unit that transmits, to the musical sound generation device, information conforming to a MIDI standard corresponding to the operating portion of which selection has been determined by the determination unit.