Gesture-Based Music Playback for Personalized Video Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multimedia platforms restrict users to creating videos based on pre-selected music, leading to insufficient personalization and poor user experience.
Innovation Solution
A music playing method and apparatus that captures user interactions in videos to determine corresponding audio processing modes, allowing users to create personalized music through actions such as touching specific interface positions or performing gestures, enabling the addition of audio tracks and special effects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If users select music from pre-provided options on multimedia platforms, then the music selection process is simple and systematic, but the personalization of created music is insufficient and user experience is poor
Solution Approach 1:
The system enables users to create music independently through their own actions and gestures without requiring complex pre-selection from provided options. Users perform actions captured by the camera, and the system automatically generates corresponding audio tracks, allowing users to serve themselves in music creation while maintaining system simplicity
Solution Approach 2:
The patent replaces the traditional mechanical music selection interface (buttons, menus, lists) with a gesture-based control system. The camera captures user actions, and computer vision algorithms interpret these actions to trigger music generation, substituting physical interaction mechanisms with optical sensing and automated processing
2Adaptability or versatility
If the system captures and analyzes user actions to generate music, then music personalization is enhanced, but the system complexity and processing requirements increase
Solution Approach 1:
The system creates a visual copy of user actions through camera capture and processes this copy to generate music. Instead of directly measuring complex physical actions, the system captures video frames, extracts action features from these frames, and uses the extracted information to control music generation, simplifying the detection process through intermediate representation
Solution Approach 2:
The system performs preliminary processing of video frames to extract action features before using them for music generation. By pre-processing the visual data to identify key action characteristics, the system reduces the complexity of subsequent music synthesis operations and prepares the data in advance for efficient processing
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
A music playing method and apparatus based on user interaction, and a device and a storage medium, which relate to the technical field of music videos. The method comprises: collecting a target video by means of a video capturing apparatus; when it is identified, in the target video, that a target object presents a target action, determining an audio processing mode corresponding to the target action; and playing music in the audio processing mode. A target video is collected by means of a video capturing apparatus; when it is identified, in the target video, that a target object performs a target action, a corresponding audio processing mode is determined on the basis of the target action; and music is played in the audio processing mode, such that a user can create personalized music at will by means of a video and according to his/her own operations, musical creation is not restricted, and the fun of musical creation is also increased, thereby improving the user experience.