Gesture-controlled music creation system in augmented and virtual reality environments
The gesture-controlled music creation system in AR and VR environments addresses the need for musical expertise by allowing users to control musical parameters through hand gestures, enhancing accessibility and interactivity in music composition.
Patent Information
- Application Number
- GB2025003348
- Authority / Receiving Office
- GB · GB
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-03-07
- Publication Date
- 2026-11-18
AI Technical Summary
Current methods of live music performance and composition require extensive knowledge of music theory and proficiency with traditional musical instruments, limiting accessibility for users with varying levels of musical expertise.
A gesture-controlled music creation system for AR and VR environments that allows users to control musical parameters such as note onset, duration, pitch, and volume through hand gestures, providing visual and tactile feedback, and enabling recording of played notes.
Simplifies music creation by making it accessible to users of all skill levels, enhancing interactivity and creativity in real-time music performance.
Abstract
Description
Background of the Invention: The advent of AR and VR technology has opened new possibilities for interactive music creation. Current methods of live music performance and composition often require extensive knowledge of music theory and proficiency with traditional musical instruments. This invention aims to simplify the music creation process by using hand gestures to control various musical parameters, thereby making it accessible to users with varying levels of musical expertise. Detailed Description: The invention is a gesture-controlled music creation system designed for use with AR and VR headsets. The system allows users to create and perform music in real-time by controlling the following parameters through hand movements: • Note Onset and Duration: Controlled by the horizontal position of the hand (x-axis). Longer notes quantized to longer bar divisions, such as whole bars, are placed when the hand is further to the left, while shorter notes, quantized to smaller subdivisions such as 1 / 16ths of bars, are placed when the hand is more towards the right. • Pitch: Controlled by the vertical position of the hand (y-axis). A pre-set musical scale ensures that the notes and chords fit within musical theory, providing a pleasing sound. • Volume: Controlled by the distance of the hand from the user (z-axis). The further the hand is from the user, the quieter the notes are played. To enable fine control: • No fingers extended: No notes are played. • One finger extended: A single note is played. • Two fingers extended: A chord is played. • Three fingers extended: A more complex chord is played. • Four or more fingers extended: Effects such as arpeggio and delay are added to the sound. The system provides visual feedback through color-coded objects superimposed on each finger in the AR environment: • Duration: Represented by the length of the object. • Pitch: Represented by the color of the object, for example, according to its chroma within an octave. For fingers not held up, this may be black to indicate this is understood by the system. A textual representation of the pitch (e.g., C#) can also be shown in a similar color in front of the user line of sight. • Volume: Represented by the opacity of the object. These objects pulse in time with the current bar and beat position, providing tactile feedback to the user. The system allows played notes to be recorded for use in larger compositions or played back later.
Claims
1. A gesture-controlled music creation system for AR and VR environments, wherein note onset and duration are controlled by the horizontal position of the hand (x-axis).
2. The system of claim 1, wherein pitch is controlled by the vertical position of the hand (y-axis) according to a pre-set musical scale.
3. The system of claim 1, wherein volume is controlled by the distance of the hand from the user (z-axis).
4. The system of claim 1, wherein no fingers extended results in no notes being played, one finger extended plays a single note, two fingers extended play a chord, three fingers extended play a more complex chord, and four or more fingers extended add effects.
5. The system of claim 1, wherein visual feedback is provided through color-coded objects superimposed on each finger, representing duration, pitch, and volume.
6. The system of claim 5, wherein the objects pulse in time with the bar and beat position.
7. The system of claim 1, wherein played notes can be recorded and played back later.
8. The system of claim 1, wherein the music is played in real time.
9. The system of claim 1, wherein the music appears on a virtual screen in real time.
Citation Information
Patent Citations
ViewWO2011/102744A1onEspacenetopensinnewtab
ViewJP2005037758AonEspacenetopensinnewtab
