Neural Network Virtual Object Movement Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for controlling the movements of a virtual object representing a performer are monotonous and stereotyped due to static associations between performance data and movement data, lacking variability in response to performance conditions like pitch and volume.
Innovation Solution
An information processing method that generates control data by analyzing time series musical notes using a trained model, combining a convolutional neural network for feature extraction and a recurrent neural network for predicting smooth movements, allowing for dynamic and varied control of the virtual object's movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If performance data and performance movement data are statically associated with each other, then the control system is simple and easy to implement, but the movements of the object become monotonous and stereotyped
Solution Approach 1:
The patent transforms the static association between performance data and movement data into a dynamic system using neural networks. The trained model dynamically generates movement data based on input performance data, allowing the system to adapt and produce varied movements while maintaining ease of operation through automated processing.
Solution Approach 2:
The patent changes the parameter representation by using neural network models that learn complex relationships between performance parameters (pitch, volume, timing) and movement parameters. This allows continuous variation in movement characteristics based on input performance data, resolving the contradiction between simplicity and variability.
2Adaptability or versatility
If a trained model is used to generate control data dynamically, then the movements of the object become varied and natural, but the device complexity increases
Solution Approach 1:
The patent applies preliminary action by training the neural network model in advance offline. The trained model is then deployed for real-time inference, where it generates varied and natural movements without requiring complex real-time processing. This separates the complexity of model training from the simplicity of real-time operation.
Solution Approach 2:
The trained neural network model acts as an intermediary between performance data and movement generation. It absorbs the complexity of learning relationships during training, while providing simple, efficient predictions during operation, thus resolving the contradiction between variability and complexity.
3Speed
If performance movement data is retrieved from storage for each note condition, then the system responds quickly to performance data, but the movements lack natural variation and expression
Solution Approach 1:
The patent replaces the mechanical lookup system (retrieving pre-defined movement data from storage based on note conditions) with a neural network-based system. This substitution maintains fast response speed through efficient neural network inference while enabling natural variation through the model's learned representations of performance-movement relationships.
Data Source
AI summary
An information processing device 11 including: a control data generation unit that inputs analysis data X that is to be processed, to a trained model that has learnt a relationship between analysis data X that represents a time series of musical notes, and control data Y for controlling movements of an object that represents a performer, thereby generating control data Y according to the analysis data X.