Gesture-Based Music Generation via Video Action Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing music creation methods on terminal devices are monotonous and lack interactivity, failing to enhance user experience and pleasure in music creation.
Innovation Solution
A method and apparatus that acquire a video of a user's actions within a preset duration, determine action information such as type and duration, and generate music based on this information, allowing users to interactively create music by performing actions that correspond to specific musical elements like scales, tones, or gestures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional piano key simulation is used for music creation, then the device complexity is low, but the interactivity and user experience are poor
Solution Approach 1:
The patent replaces the mechanical interaction of touching piano keys with gesture recognition technology. The system captures video of user gestures, processes them through computer vision algorithms, and converts them into musical notes. This substitution of mechanical interaction with automated recognition systems resolves the contradiction by providing rich interactivity through diverse gestures while keeping the physical device simple.
Solution Approach 2:
The patent introduces an intermediary system consisting of video capture, gesture recognition algorithms, and music generation modules. This intermediary layer translates physical gestures into musical expressions, enabling sophisticated interactivity without requiring complex physical interfaces. The intermediary processing system bridges the simple input (gestures) and the complex output (music generation).
2Ease of operation
If gesture recognition is implemented for music creation, then the interactivity and user pleasure are enhanced, but the processing complexity increases
Solution Approach 1:
The system employs self-service mechanisms where the gesture recognition algorithm automatically identifies and interprets user gestures without requiring manual input or configuration. The music generation process is also automated, converting recognized gestures directly into musical compositions. This automation reduces the operational burden on users while managing processing complexity through efficient algorithm design.
Solution Approach 2:
The patent utilizes parameter changes in gesture characteristics (position, velocity, acceleration, duration) to generate varied musical expressions. By monitoring and responding to changes in these parameters, the system provides rich interactive experience without requiring complex processing of every possible gesture variation, as it focuses on key parameter thresholds and patterns.
Data Source
AI summary
A method for generating music and apparatus, a device, an electronic device, a computer readable storage medium, a computer program product, and a computer program. The method comprises: acquiring a first video of first user captured within a preset duration; determining, action information of a first action performed by the first user within the preset duration according to the first video, the action information comprising an action type and a duration of action; and then generating, according to the action information of the first action, target music corresponding to the preset duration. By means of the process, a user can interact with a terminal device by performing a series of actions to achieve music creation, which improves the interactivity and fun of the music creation process, thereby improving user experience.