Wearable Sensor-Based Automatic Soundtrack Generation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Creating a soundtrack for user-generated videos is time-consuming and requires specific skills, making it difficult for everyday users to produce an attractive soundtrack quickly and easily, especially with the limitations of existing tools that still require manual effort and software expertise.

Innovation Solution

A system comprising a camera device and wearable sensors that generate a soundtrack automatically by processing data from sensors like accelerometers, GSR, and EEG during video recording, allowing for real-time soundtrack creation without manual intervention or complex software knowledge, synchronizing music properties with video events and physical properties.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If existing tools like XHAIL or MOODELIZER are used to create soundtracks, then the soundtrack creation process becomes easier for amateurs, but it still requires hours or days of manual work and specific skill sets

Engineering Contradiction:
Improveease of soundtrack creationVSAvoidtime required for soundtrack creation
Core Design Contradiction:
Ease of operationVSLoss of time

Solution Approach 1:

The system enables automatic soundtrack generation by processing sensor data from the user's own device without requiring manual intervention. The control unit automatically analyzes accelerometer, GSR, and EEG data to generate appropriate soundtrack segments, eliminating the need for users to manually edit or select music

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The patent replaces manual soundtrack creation processes with automated computational analysis of physiological sensor data. Instead of requiring users to manually select and edit music based on their understanding of composition, the system uses algorithms to automatically match soundtrack properties with measured physical properties from sensors

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Ease of operation

If manual soundtrack creation is performed by users without specialized skills, then the process becomes accessible to everyday users, but the quality and attractiveness of the soundtrack suffers

Engineering Contradiction:
Improveaccessibility to everyday usersVSAvoidsoundtrack quality
Core Design Contradiction:
Ease of operationVSManufacturing precision

Solution Approach 1:

The system continuously monitors sensor data during video recording and uses this real-time feedback to dynamically adjust and select appropriate soundtrack segments. The control unit processes ongoing measurements of acceleration, galvanic skin response, and brain activity to automatically synchronize music selection with the user's actual physical state and video events

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The system changes the parameters used for soundtrack selection from subjective user judgment to objective physiological measurements. By basing music selection on quantifiable sensor data such as heart rate variability, acceleration patterns, and brain wave activity, the system achieves professional-quality soundtrack matching without requiring user expertise in music composition

Inventive Principle:
Principle #35Parameter changes

3Manufacturing precision

If users spend time learning complicated software and video editing skills, then they can create better soundtracks, but this time investment reduces the overall efficiency of video production

Engineering Contradiction:
Improvesoundtrack qualityVSAvoidoverall video production efficiency
Core Design Contradiction:
Manufacturing precisionVSProductivity

Solution Approach 1:

The system performs soundtrack creation autonomously without requiring user intervention or learning. The control unit automatically processes sensor data and generates the complete soundtrack, freeing users to focus on video content creation rather than technical editing skills

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The system collects and processes sensor data during the video recording itself, performing soundtrack preparation in advance rather than requiring post-production editing. This preliminary action eliminates the need for separate learning and editing phases, improving overall production efficiency

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS10902829B2Method and system for automatically creating a soundtrack to a user-generated video
Publication Date: 2021.01.26 SONY GROUP CORP
  • US10902829B2 patent drawing
  • US10902829B2 patent drawing

AI summary

The invention relates to a system for automatically creating a soundtrack, comprising a camera device (1, 1′) for recording a user-generated video, at least one wearable sensor (3, 3′), and a control unit (2, 2′) in communication with the camera device (1, 1′) and the at least one wearable sensor (3, 3′). The control unit (2, 2′) is adapted to generate the soundtrack based on data gathered from the at least one wearable sensor (3, 3′) during the recording of the user-generated video. The invention further relates to a method for automatically creating a soundtrack, computer program product, a computer readable memory storage unit computing arrangement or mobile device (1, 11) for executing the method.