Music generation method of intelligent wearable device and terminal

Abstract

The invention discloses a music generation method of an intelligent wearable device and a terminal, and the method comprises the steps: obtaining motion physiological data captured by the intelligent wearable device through at least two sensors, and carrying out the fusion of the motion physiological data captured by each sensor, and obtaining motion feature information, so as to capture the limb motion of a user in real time; converting the action feature information into a music control signal based on the historical music generation data to generate a personalized music signal; the music control signal is mapped to the music element of the preset music library, and the action music corresponding to the captured action physiological data is generated according to the music element obtained through mapping, so that the real-time association between the user limb action and music playing is realized, the diversity of music generation is improved, and the interactivity between the user action and the music is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of audio processing, and particularly to a music generation method and a terminal for intelligent wearable devices. Background Art

[0002] In the prior art, some music devices based on somatosensory interaction technology have been applied. Somatosensory music devices usually capture the user's body movements through motion capture devices (such as Kinect, Leap Motion, etc.) or wearable devices (such as smart bracelets, smart watches), and then control the generation of preset music or sound effects. For example, some devices use acceleration sensors or gyroscopes to convert the user's actions such as waving hands and jumping into digital signals for controlling music players or triggering sound effects.

[0003] At present, most of the music devices based on somatosensory interaction on the market are independent hardware devices, which are large in size, not convenient for daily wearing and carrying, and difficult to achieve the convenience and interactivity of real-time performance. Moreover, the action recognition of the prior art often depends on specific capture devices, such as external cameras, somatosensory controllers, etc., which limits the user's activity range and freedom, and it is difficult to perform complex action performances. However, existing wearable music devices usually fail to integrate multiple sensors and cannot comprehensively detect different types of user actions (such as rotation, bending, pressing, etc.), thus limiting the diversity of music generation and detailed control.

[0004] Many prior arts can only simply control the play, pause or sound effect switching of music, lack complex music generation mechanisms, and are difficult to generate real-time dynamic music according to the user's actions. The interactivity between music and actions is low, and precise real-time music control cannot be achieved. The interaction methods of existing devices are mostly simple matches of preset fixed actions and sound effects, lacking innovative and in-depth user experiences. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: to provide a music generation method and a terminal for intelligent wearable devices, which can improve the diversity of music generation and enhance the interactivity between user actions and music.

[0006] To solve the above technical problem, the technical solution adopted by the present invention is: A music generation method for an intelligent wearable device, comprising the steps of: S1. Obtain the action physiological data captured by the intelligent wearable device through at least two sensors, and fuse the action physiological data captured by each sensor to obtain action feature information; S2. Convert the action feature information into a music control signal based on historical music generation data; S3. Map the music control signal to the music elements in a preset music library, and generate action music corresponding to the captured action physiological data based on the mapped music elements.

[0007] To solve the above technical problems, another technical solution adopted by the present invention is: A music generation terminal for an intelligent wearable device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements each step of the above-mentioned music generation method for an intelligent wearable device when executing the computer program.

[0008] The beneficial effects of the present invention are as follows: Obtain the action physiological data captured by an intelligent wearable device through at least two sensors, fuse the action physiological data captured by each sensor to obtain action feature information, so as to capture the user's limb movements in real time; convert the action feature information into a music control signal based on historical music generation data to generate personalized music signals; map the music control signal to the music elements in a preset music library, and generate action music corresponding to the captured action physiological data based on the mapped music elements, thereby realizing the real-time association between the user's limb movements and music performance, improving the diversity of music generation, and enhancing the interaction between the user's actions and music. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a flowchart of a music generation method for an intelligent wearable device according to an embodiment of the present invention; Figure 2 It is a schematic diagram of a music generation terminal for an intelligent wearable device according to an embodiment of the present invention.

[0010] LABEL DESCRIPTION: 1. A music generation terminal for an intelligent wearable device; 2. Memory; 3. Processor. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0011] To describe the technical content, achieved objectives and effects of the present invention in detail, the following is described in conjunction with the embodiments and with reference to the accompanying drawings.

[0012] Please refer to Figure 1 , an embodiment of the present invention provides a music generation method for an intelligent wearable device, including the steps of: S1. Obtain the action physiological data captured by an intelligent wearable device through at least two sensors, and fuse the action physiological data captured by each sensor to obtain action feature information; S2. Convert the action feature information into a music control signal based on historical music generation data; S3. Map the music control signal to the music elements in a preset music library, and generate action music corresponding to the captured action physiological data according to the mapped music elements.

[0013] As can be seen from the above description, the beneficial effects of the present invention are as follows: Obtain the action physiological data captured by at least two sensors of the smart wearable device, fuse the action physiological data captured by each sensor to obtain action feature information, so as to capture the user's body movements in real time; Based on the historical music generation data, convert the action feature information into a music control signal to generate a personalized music signal; Map the music control signal to the music elements in a preset music library, and generate action music corresponding to the captured action physiological data according to the mapped music elements, thereby realizing the real-time association between the user's body movements and music performance, improving the diversity of music generation, and enhancing the interaction between the user's actions and music.

[0014] Further, in step S1, fusing the action physiological data captured by each sensor to obtain action feature information includes: Respectively use Kalman filter, particle filter and extended Kalman filter to perform state estimation on the action physiological data captured by each sensor; Perform weighted average or optimal fusion on each state estimation result to obtain action feature information.

[0015] As can be seen from the above description, respectively use Kalman filter, particle filter and extended Kalman filter to perform state estimation on the action physiological data, and perform weighted average or optimal fusion on each state estimation result to improve the accuracy and robustness of data fusion.

[0016] Further, step S2 includes: Obtain historical music generation data, extract the music features of the historical music generation data and analyze the historical performance mode of the corresponding user; Train a performance mode recognition model according to the music features of the historical music generation data and the historical performance mode of the corresponding user, so as to recognize the user's real-time performance mode in real time according to the performance mode recognition model; Generate a music control signal according to the user's real-time performance mode and the action feature information.

[0017] As can be seen from the above description, combine the historical music generation data and its historical performance mode to train the performance mode recognition model, so as to facilitate the recognition of the user's real-time performance mode. In this way, a music control signal can be generated according to the user's real-time performance mode, making the generation of the music control signal more dynamic and personalized.

[0018] Further, generating a music control signal according to the user's real-time playing mode and the action feature information includes: Determining the rhythm of the music control signal by combining the real-time playing mode and the action frequency of the action feature information; Analyzing the user's emotional characteristics according to the intensity information of the action feature information, and determining the timbre of the music control signal according to the user's emotional characteristics and the real-time playing mode; Determining the notes of the music control signal by identifying the gesture features of the action feature information.

[0019] As can be seen from the above description, the generation of the music signal is adjusted according to information such as the real-time playing mode and the user's emotional characteristics. It can be seen that the generation of the music signal not only depends on the real-time action data, but also adds more context information (such as the user's playing habits, emotional tendencies, current environment, etc.), making the generation process more flexible and having real-time feedback.

[0020] Further, after step S3, it further includes: Capturing real-time action physiological data, and adjusting the rhythm, volume and pitch of the action music according to the emotional characteristics and action features in the real-time action physiological data.

[0021] As can be seen from the above description, dynamically adjusting the rhythm, volume and pitch of the action music can provide more personalized and natural audio feedback, adapting to different users' emotional and behavioral patterns.

[0022] Please refer to Figure 2 , another embodiment of the present invention provides a music generation terminal for an intelligent wearable device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements each step of the above-mentioned music generation method for an intelligent wearable device.

[0023] The above-mentioned music generation method and terminal for an intelligent wearable device of the present invention are applicable to improving the diversity of music generation and enhancing the interaction between the user's actions and music. The following is illustrated by specific embodiments: Please refer to Figure 1 , Embodiment 1 of the present invention is: A music generation method for an intelligent wearable device. In this embodiment, the intelligent wearable device is preferably an intelligent vest, and includes the steps of: S1. Obtaining action physiological data captured by the intelligent wearable device through at least two sensors, and fusing the action physiological data captured by each sensor to obtain action feature information.

[0024] S11. Obtain the motion physiological data captured by the smart wearable device through at least two sensors. Specifically, a variety of sensors are integrated inside the smart wearable device, which can capture user data in different dimensions.

[0025] Among them, the sensors include an acceleration sensor for detecting linear motions such as waving and jumping, a gyroscope for detecting angular changes such as rotation and torsion, and a pressure sensor for monitoring the force of the user's pressing or contact. In addition to inertial sensors (such as accelerometers and gyroscopes), it can also include a bioelectrical signal sensor and an environmental sensor. The bioelectrical signal sensor is used to obtain the physiological state of the user, and can be an optional heart rate monitoring sensor or an electromyogram (EMG) sensor; the environmental sensor is used to sense the influence of the external environment on the user's actions, and then optimize action recognition and signal generation, and can be an optional temperature sensor, humidity sensor, and pressure sensor.

[0026] These sensors can further capture more detailed data such as the intensity, speed, and direction change of the user's actions. Therefore, the data sources are diverse and can provide richer input signals.

[0027] Furthermore, it is also necessary to perform real-time preprocessing on the multi-dimensional data input by the sensors, such as steps like noise removal, smoothing filtering, and data normalization, to ensure the accuracy and real-time nature of the signals.

[0028] S12. Fuse the motion physiological data captured by each sensor to obtain motion feature information.

[0029] In this embodiment, there is a multi-sensor integration module, which is used to collect data from different sensors and integrate the multi-source sensor data using Kalman filtering or other fusion algorithms, so as to obtain a more accurate user motion state.

[0030] In the process of data fusion, in addition to using Kalman filtering (for data fusion in dynamic systems), advanced algorithms such as particle filtering and extended Kalman filtering can be combined. Especially in non-linear or complex environments, state estimation can be performed more precisely. In some embodiments, Kalman filtering, particle filtering, and extended Kalman filtering can be used respectively to perform state estimation on the motion physiological data, and the respective state estimation results can be weighted averaged or optimally fused to improve the accuracy and robustness of data fusion.

[0031] A Bayesian network probability model can also be introduced in the process of data fusion to effectively handle the uncertainty of data. Especially when facing multiple uncertain sensor data sources, the decision-making process can be optimized.

[0032] In some embodiments, models such as deep neural networks (DNNs) and convolutional neural networks (CNNs) can also be introduced to automatically learn features from complex multi-dimensional sensor data to achieve more accurate action recognition and prediction. Reinforcement learning methods can also be combined to dynamically adjust the model and parameters through the user's real-time action feedback, so as to generate more personalized and real-time music control signals subsequently.

[0033] S2. Convert the action feature information into a music control signal based on historical music generation data.

[0034] S21. Obtain historical music generation data, extract the music features of the historical music generation data, and analyze the historical playing patterns of the users corresponding to the historical music generation data.

[0035] Specifically, first, collect the user's historical music generation data and preprocess the data: The historical music generation data can include different types of sensor data (such as accelerometers, gyroscopes, pressure sensors, etc.), and can also include the interaction logs between the user and the device (such as actions, rhythms, notes, etc.). Among them, it is necessary to process the noise in the original data and remove unnecessary signals (such as environmental noise) to ensure the accuracy of the data. Common denoising methods include Kalman filtering, moving average, etc. Then, label the data according to the user's playing style and needs. The data labeling can be classified according to different playing types (such as playing styles, rhythm speeds, note pitches, etc.). Normalize the sensor data from different sources so that the data is in the same scale range. For example, normalize all sensor data to the interval of 0-1 to ensure the consistency of data processing.

[0036] Then, perform music feature extraction and playing pattern analysis on the historical music generation data: Extract motion features related to the historical music generation data, such as hand position changes, wrist rotation angles, pressure changes, speeds, etc. Extract features such as time and frequency according to the action type, such as the duration of each play, starting speed, strength, etc. Calculate the rhythm and beat points of the historical music generation data, and extract information such as the duration and interval of each note to analyze the user's playing habits (such as whether they tend to play fast or slow rhythms, whether there are pauses, etc.). Based on the changes in strength and rhythm fluctuations in the user's playing habits, analyze the user's emotional tendencies. For example, the strength and rhythm of playing may be related to emotions (such as joy, sadness, etc.), and extract such emotional features for subsequent music generation.

[0037] At the same time, analyze the user's playing style in different time periods through time series analysis (such as dynamic time warping (DTW), Hidden Markov Model (HMM), etc.) to identify their common playing patterns.

[0038] S22. Train a performance pattern recognition model based on the music features of the historical music generation data and the historical performance patterns of the corresponding users, so as to recognize the real-time performance patterns of users in real time according to the performance pattern recognition model.

[0039] Specifically, first, based on the historical music generation data, construct a training set and a test set. The training set contains different performance segments of users and style labels of performance patterns (such as genres, emotion categories, etc.), and the test set is used to evaluate the accuracy of the model.

[0040] Secondly, use machine learning algorithms (such as decision trees, support vector machines (SVM), or neural networks, etc.) to train the extracted features. The objective function can be adjusted according to different tasks (such as emotion recognition, style classification, etc.) during the training process.

[0041] Then, use deep learning models (such as LSTM, CNN, etc.) to train the historical music generation data. Among them, LSTM (long short-term memory network) is particularly suitable for processing time series data and can help recognize the rhythm, emotional changes, etc. of the user's performance.

[0042] Finally, through the trained model, the user's preferred performance pattern can be recognized in real time, and it can be recognized whether the user is inclined to a certain specific style (such as classical, rock, jazz, etc.) or has some habitual performance patterns.

[0043] S23. Generate a music control signal according to the user's real-time performance pattern and the action feature information.

[0044] Specifically, adjust the music generation model according to the user's preferred performance pattern. For example, for users with a preferred performance pattern tending to slow rhythms, generate a softer and more melodious music control signal, while for users with a preferred performance pattern tending to fast rhythms, generate a faster and more dynamic music control signal. In some embodiments, continuous learning and optimization can be carried out during the actual performance. As the user's performance data accumulates, the model is continuously adjusted to adapt to the changes of the user.

[0045] In this embodiment, the music rhythm and timbre can be adjusted according to the user's preferred performance pattern and action feature information, and are mapped to the music generation model in real time to generate a music control signal. The specific steps are as follows: S231. Determine the rhythm of the music control signal by analyzing the action frequency of the user's action feature information (such as key pressing force, hand movement speed, etc.). Based on the user's historical music generation data, the model will judge whether the user tends to a faster rhythm or a slower rhythm. Map the user's actions to the rhythm in the music according to the user's rhythm preference. For example, if the user tends to play quickly, the rhythm speed of the music will be automatically increased, and vice versa. If there are rhythm fluctuations during the user's performance (for example, a certain passage suddenly speeds up or slows down), the rhythm of the background music can also be adjusted according to the rhythm fluctuations to ensure that the music is synchronized with the user's actions.

[0046] S232. Adjust the timbre of the music control signal according to the force information of the user's action feature information (such as the force of key pressing, the force of actions, etc.). Strong actions can trigger a more plump and rich timbre, while gentle actions correspond to a gentle timbre. Adjust the emotional expression of the timbre by analyzing the emotional characteristics of the user's action feature information (for example: the pressure fluctuations and force changes when the user is performing). For example, a strong performance can generate a more aggressive timbre, while a gentle performance can generate a more gentle timbre.

[0047] According to different preferred performance modes (such as classical, rock, etc.), the timbre of the music can be adjusted to generate the timbre of the corresponding style. For example: for the rock style, the electric timbre of the guitar may be used, while for the classical style, the timbre of the piano or string instruments may be used.

[0048] S233. Map the user's action feature information (such as gestures, finger presses) to specific notes or melody passages. For example, a certain gesture can trigger notes in the high pitch range, while another gesture triggers notes in the low pitch range; based on the recognition of gestures, the notes suitable for the action can be automatically selected, and the duration and volume of the notes can be controlled according to the duration and force of the gestures.

[0049] S3. Map the music control signal to the music elements in the preset music library, and generate the action music corresponding to the captured action physiological data according to the mapped music elements.

[0050] In this embodiment, the preset music library contains accompaniments and melodies of various styles. The user's action signals correspond one by one to these music segments, forming a dynamic music generation mechanism. The music control signal corresponding to each action triggers one or more music segments, and the playing order and changes of the music are determined according to parameters such as the order, rhythm, and force of the actions; corresponding music segments are generated according to parameters such as the intensity, direction, and speed of different actions, ensuring the precise matching and natural transition between the actions and the music. Thus, action music is generated in real time, realizing the real-time music performance function.

[0051] S31. The process of dynamically generating the most matching music segments based on the real-time data of multi-dimensional sensors and combining machine learning algorithms is divided into several steps. This process includes data acquisition, feature extraction, model training, and real-time inference. The goal is to extract features from the user's motion data through machine learning algorithms (such as KNN, neural networks, etc.) and generate matching music segments.

[0052] Specifically, the motion data of the user is collected in real time through multiple sensors (such as accelerometers, gyroscopes, electromyogram (EMG) sensors, pressure sensors, etc.). The data output by each sensor contains information such as the direction, speed, intensity, acceleration, and rotation angle of the motion. The original sensor data usually contains noise or incomplete information, so data cleaning and preprocessing are required. Common steps include: using filters (such as low-pass filters or Kalman filters) to remove high-frequency noise; normalizing the data of different sensors in a certain proportion to make them have similar dimensions and ranges; extracting useful features from the original data, such as: the amplitude, speed, acceleration, rotation angle, and angular velocity of the motion (such as using gyroscope data), the electromyogram activity signal (such as using an electromyogram (EMG) sensor to capture muscle movement), the frequency and directionality of the motion, etc.

[0053] By analyzing a large amount of motion data, features related to music generation are selected. For example, certain motions may affect the timbre (such as changes in force affecting pitch), while other motions may affect the rhythm or melody changes. The selected features include: static features: such as the average amplitude and maximum acceleration of the motion; dynamic features: such as the rate of change of the motion and the time series of speed and acceleration; interaction features: combined features of multiple sensor data (such as the combination of rotation and acceleration).

[0054] In the training stage, labeled training data is used to train the machine learning model to learn the mapping relationship from user motions to music segments. Common methods include: K-Nearest Neighbors (KNN): For the given real-time motion data, the KNN algorithm calculates the similarity between the input data and the historical training data, finds the K closest motions, and makes predictions based on the labels of these neighbors (i.e., the corresponding music segments).

[0055] Neural Network: For more complex mapping relationships, neural networks (such as feedforward neural networks or convolutional neural networks) can learn the complex mapping relationship from multi-dimensional sensor data to music through multi-layer non-linear transformations. Generally, a neural network model will have: Input layer: Receives data features from multi-dimensional sensors (such as acceleration, angle, electromyogram signals, etc.); Hidden layer: Multiple hidden layers perform non-linear transformations on the input to extract high-level features; Output layer: Generates features corresponding to music segments, such as pitch, timbre, rhythm, etc. The output can be the probability distribution of specific music segments, or the features of the corresponding music segments (such as spectrum, rhythm, etc.).

[0056] Through a labeled training dataset, the user's motion data is paired with the corresponding music segments. The training data can be constructed by collecting the user's performance data and the corresponding music data. Usually, a large number of sample data are required to capture different performance styles and emotional changes. Use traditional machine learning training methods (such as KNN) or deep learning methods (such as neural networks) to train the model. The training process adjusts the model parameters by optimizing the loss function (such as classification error, regression error) so that the model can accurately predict the music segment that best matches the given motion.

[0057] During runtime, the user's motion is collected in real time by sensors and fed into the trained machine learning model. The model extracts features and makes real-time inferences based on the current motion data (such as speed, acceleration, angle, pressure, etc.). For real-time input motion data, the KNN algorithm calculates the distance from the historical training data and selects the K most similar training samples, and selects the corresponding music segments to generate real-time feedback; the neural network model will generate corresponding music segments according to the input feature data. For example, it can generate a change in a note, a rhythm, or a timbre. The output of the neural network can be the frequency, pitch, duration of the note, or the parameters controlling music playback (such as rhythm, volume, etc.).

[0058] During the user's performance, dynamically adjust the generated music according to the music control signal to ensure the synchronization of the music with the performance motion. Based on the adjustment of timbre and rhythm, generate real-time audio output through the audio processing unit and play it to the user through headphones or other audio devices.

[0059] S32. Perform double optimization of the generated music signal in the time domain and frequency domain, adjust the rhythm, volume and pitch of the music through algorithms to ensure its high degree of fit with the user's motion and emotional state. Introduce an adaptive algorithm to dynamically adjust the audio parameters according to the changes of different users and environments, making the effect of the music more natural and personalized.

[0060] In some embodiments, the timbre, rhythm, and note mapping can also be gradually adjusted according to the performance effects feedback by the user (such as whether it feels comfortable, whether it meets expectations, etc.). Specifically, after generating a music segment, the emotion and style of the music can be adjusted by modulating the output of the model or adding post-processing algorithms. For example, by fine-tuning the timbre, speed, intensity, etc. of the notes to make them conform to the user's current performance style or emotional needs. To improve the intelligence of the terminal, an online learning mechanism can be introduced in real-time applications. The model can be continuously optimized according to the actual feedback of the user (such as changes in the performance style) to make it more adaptable to the user's actions and music generation.

[0061] In addition to KNN and neural networks, the following several methods are also used to further improve the performance of the model: Sequence models (LSTM / GRU): For tasks involving temporal features, using recurrent neural networks (RNNs), especially long short-term memory networks (LSTMs) or gated recurrent units (GRUs), can help the model capture the temporal dependencies of the action data. This can better handle the dynamic relationships between actions and make the generated music segments smoother and more natural in terms of rhythm and timbre changes.

[0062] Reinforcement Learning: Reinforcement learning can be used to optimize the quality of the generated music segments. In this setting, the generated music segments will be adjusted according to the real-time feedback of the user, and the generation strategy will be continuously improved according to the user's preferences (such as through reward signals).

[0063] Generative Adversarial Networks (GAN): For more complex generation tasks, generative adversarial networks (GANs) can be used to generate more realistic and natural music segments. GANs make the generated music more realistic by having the generator and discriminator play against each other.

[0064] By combining the data of multi-dimensional sensors and machine learning algorithms (such as KNN, neural networks, etc.), the goal of dynamically generating the most matching music segments according to the user's real-time action data can be achieved. Machine learning methods can establish a non-linear mapping relationship between processing user actions and generating music signals, and can generate music that conforms to the user's action characteristics in real-time. By improving the algorithms (such as using LSTM, reinforcement learning, or GAN, etc.), the performance and generation effect of the terminal can be further improved.

[0065] In the real-time audio processing module, the rhythm, volume, and pitch of the music are dynamically adjusted according to the user's emotional state or action intensity. This adjustment process can be based on static mapping or on dynamic feedback and machine learning algorithms for real-time adjustment. Two possible adjustment methods are described in detail below: static mapping and dynamic adjustment.

[0066] 1. Static mapping method Static mapping associates sensor data with audio attributes (such as rhythm, volume, pitch, etc.) through predefined rules. This method is simple and intuitive but has low flexibility and is usually applicable to some basic application scenarios. The specific process is as follows: Bio-signal acquisition: Real-time physiological data of the user is obtained through bio-signal sensors (such as heart rate monitoring, galvanic skin response, electromyogram signals, etc.).

[0067] Emotional state detection: For example, an accelerated heart rate and an increased galvanic skin response may indicate that the user is in an excited state, while a low heart rate or stable electromyogram signals may indicate relaxation or calmness.

[0068] Action intensity detection: The action intensity of the user (such as acceleration, angle change, speed, etc.) can also be used as a basis for adjusting the audio.

[0069] Definition of static mapping table: According to previous research or experimental data, a set of static mapping tables are created to fix the relationship between sensor data and audio features. For example: The relationship between heart rate and rhythm: When the heart rate speeds up, the rhythm (BPM) of the music accelerates, and when the heart rate drops, the rhythm slows down. The relationship between galvanic skin response and volume: When the galvanic skin response increases, the volume of the music increases, and vice versa. The relationship between action intensity and pitch: When the amplitude or force of the user's action increases, the pitch can be raised; when the action weakens, the pitch decreases.

[0070] Audio adjustment: According to the real-time input of bio-signals, the corresponding audio adjustment values are found through the static mapping table, and the rhythm, volume, and pitch of the audio are adjusted. For example, if the heart rate increases, the rhythm of the music can be increased by looking up the heart rate mapping table; if the action intensity increases, the volume will be increased.

[0071] 2. Dynamic adjustment method The dynamic adjustment method uses more complex algorithms (such as machine learning, real-time feedback mechanisms) to dynamically adjust the rhythm, volume, and pitch of music according to the user's real-time data (such as emotional state, action intensity, etc.). This method can respond to the user's changes more intelligently and personalizedly.

[0072] Real-time data acquisition and preprocessing: Similar to the static mapping method, first, the user's bio-signal data (such as heart rate, electromyogram, acceleration, galvanic skin response, etc.) is collected through sensors and real-time preprocessing (such as filtering, denoising, etc.) is performed to ensure the accuracy of the data.

[0073] Feature Extraction and Emotion / Actions Pattern Analysis: After real-time preprocessing, some sentiment analysis and action recognition algorithms (such as machine learning-based classification or regression models) can be used to extract key features from biological signals. For example: Sentiment analysis models (such as models based on heart rate and galvanic skin response): By analyzing the user's physiological data, predict the user's current emotional state (such as excited, calm, anxious, etc.). Actions pattern analysis: Infer the current action type (such as intense actions, gentle actions, etc.) based on features such as the intensity, speed, and direction of the action.

[0074] Real-time Decision Adjustment: Based on the results of emotion and actions pattern analysis, the audio can be dynamically adjusted in the following ways: Rhythm Adjustment: If it is detected that the user is excited or the action intensity increases, the rhythm (BPM) of the music can be adjusted to match the current emotion and action intensity. For example, fast actions or high-intensity emotions (such as anxiety, pleasure) can accelerate the rhythm of the music, and vice versa.

[0075] Volume Adjustment: If it is detected that the user's emotional state becomes excited or the action becomes intense, the volume can be increased to enhance the immersion; if the user's emotion tends to be calm or the action weakens, the volume can be decreased.

[0076] Pitch Adjustment: According to the user's action intensity or emotional changes, the pitch (tone height) can also be dynamically adjusted. For example, the pitch may increase when the emotion is excited and decrease when calm; large actions may cause the pitch to rise, while gentle actions may cause the pitch to drop.

[0077] Real-time Feedback and Optimization: To ensure that the audio adjustment effect is natural and closely matches the user's behavior, the adjustment strategy can be continuously optimized. For example, using reinforcement learning methods in machine learning, continuously adjusting the model parameters according to the user's feedback (such as whether they like the current sound effect or are satisfied), so that the audio adjustment better conforms to the user's preferences.

[0078] The audio adjustment is carried out in two ways: The static mapping method relies on preset rules, while the dynamic adjustment method relies on real-time analysis of the user's biological signals and action data, combined with machine learning models for more intelligent audio adjustment. The static mapping method is relatively simple and direct, but less flexible; while the dynamic adjustment method is more complex and can provide more personalized and natural audio feedback, adapting to different users' emotional and behavioral patterns.

[0079] In some embodiments, users can use the APP to monitor in real time the relationship between their movements and music output, and support dynamically adjusting mapping rules and parameters during the performance, such as adjusting the sensitivity of rhythm response, changing the generation logic of melodies, etc. In addition, visual feedback (such as LED lights) can be provided to show different colors and lighting effects according to the user's movements. Users can also ensure the expansion and optimization of functions by regularly updating the APP or firmware. It supports the loading of new music libraries and the optimization of motion recognition algorithms to adapt to different performance requirements. Through these steps, the intelligent music vest can successfully achieve real-time music performance based on movements, providing users with a brand-new music interaction experience, with high operability, entertainment value, and application value.

[0080] The specific application scenarios of this embodiment include but are not limited to: (1) In concerts or stage performances, performers wearing the intelligent music vest can play music in real time through body movements, enhancing the performance effect. The audience can not only hear the music but also see how the performers' movements directly affect the rhythm and melody of the music, increasing the interactivity and visual impact of the performance. The real-time feedback mechanism of the intelligent vest can also be demonstrated through the coordination of LED lighting effects and movements, making the performance more dynamic and technological.

[0081] (2) In music education, the intelligent music vest can help students experience the relationship between music rhythm and melody through movements, stimulating students' interest in music. Especially in music enlightenment education, students can control notes and chords through simple movements, intuitively feel the structure and changes of music, and reduce the dependence on instrument skills. In addition, the feedback device of the intelligent music vest can also help teachers monitor students' progress in real time and provide targeted guidance.

[0082] (3) In gyms or personal workouts, the intelligent music vest can add an interactive music function to the fitness process. Users can trigger different music rhythms and melodies through exercise movements, making the exercise process more interesting and dynamic. In addition, in interactive entertainment activities such as parties or team competitions, users can wear the intelligent music vest and add entertainment and novelty to social occasions through the interaction between movements and music.

[0083] (4) In science and technology art exhibitions, the intelligent music vest can be used as part of an interactive art installation. Audience members can wear the vest and use body movements to generate music and sound effects, participating in the creation of artworks. The exhibition venue can set corresponding music styles and interaction rules according to different exhibits, giving the audience the right to participate in the performance and enhancing the interaction and immersion between the audience and the artworks.

[0084] (5) In brand promotion activities, the intelligent music vest can help enterprises showcase the combination of creativity and technology of their products. Models or promotional staff wearing the intelligent vests play the brand-customized background music or sound effects through body movements, attracting the attention of the audience. Enterprises can also design exclusive interactive experiences to increase the fun and uniqueness of brand promotion, making consumers have a deeper impression of the brand during the process of participating in the interaction.

[0085] Please refer to Figure 2 , Embodiment 2 of the present invention is: A music generation terminal 1 of an intelligent wearable device, comprising a memory 2, a processor 3, and a computer program stored on the memory 2 and executable on the processor 3. When the processor 3 executes the computer program, each step of a music generation method of an intelligent wearable device in Embodiment 1 is implemented.

[0086] In summary, for the music generation method and terminal of an intelligent wearable device provided by the present invention, action physiological data captured by the intelligent wearable device through at least two sensors is obtained, and the action physiological data captured by each sensor is fused to obtain action feature information to capture the user's body movements in real time; based on historical music generation data, the action feature information is converted into a music control signal to generate a personalized music signal; the music control signal is mapped to music elements in a preset music library, and action music corresponding to the captured action physiological data is generated according to the mapped music elements, thereby realizing the real-time association between the user's body movements and music performance, improving the diversity of music generation, and enhancing the interactivity between the user's actions and music.

[0087] The intelligent wearable device of the present invention enables users to easily play music through body movements by means of somatosensory interaction technology, reducing the complexity of traditional musical instrument performance and greatly enhancing the fun and interactivity of music performance. Users can participate in music creation without mastering complex musical instrument skills, increasing the freedom and expressiveness of music performance. It can adapt to various application scenarios such as stage performances, music education, and rehabilitation training. Whether it is used for complex stage performances or relaxed entertainment activities, it can dynamically adjust the music generation and feedback mechanisms according to the usage scenario to ensure diverse and highly adaptable user experiences. It provides a brand-new tool for music education and rehabilitation training. Through the somatosensory interaction method, students can intuitively understand the rhythm and melody of music through actions, helping them better master music knowledge. In addition, in the field of rehabilitation, the intelligent music vest can combine music therapy to improve the body coordination and perception ability of patients and promote the rehabilitation effect.

[0088] Moreover, the smart wearable device of the present invention breaks the single form of traditional music performance, enabling richer interaction between performers and the audience. The audience can not only watch the performers' movements but also feel the direct relationship between the movements and the music, increasing the sense of participation and viewing pleasure. The performers play music through their movements, making the performance more expressive and immersive. It provides a new interactive expression way for music creation and performance. By combining motion capture and music generation technologies, performers can instantly create music through their movements, breaking the limitations of traditional performances, expanding the boundaries of music creation, and inspiring more possibilities for artistic innovation.

[0089] The smart wearable device of the present invention can not only generate music but also combine with a visual feedback system (such as LED lighting effects) to synchronously display dynamic light effects according to the performers' movements, enhancing the visual impact and viewing pleasure of the performance effect and further enriching the performance forms of music performances.

[0090] The above are only embodiments of the present invention and do not limit the patent scope of the present invention accordingly. Any equivalent transformation made by using the content of the specification and drawings of the present invention, directly or indirectly applied in the relevant technical fields, shall be included in the patent protection scope of the present invention by the same token.

Claims

1. A music generation method for a smart wearable device, characterized in that: Includes steps: S1. Acquire motion physiological data captured by at least two sensors of the smart wearable device, fuse the motion physiological data captured by each sensor, and obtain motion feature information; S2, converting the action feature information into a music control signal based on historical music generation data; S3. Mapping the music control signal to music elements of a preset music library, and generating action music corresponding to the captured action physiological data according to the mapped music elements.

2. The music generation method of a smart wearable device according to claim 1, characterized in that: In step S1, the motion physiological data captured by each sensor are fused to obtain motion feature information, including: Kalman filter, particle filter and extended Kalman filter are used to estimate the state of the action physiological data captured by each sensor; The state estimation results are weighted averaged or optimally fused to obtain the motion feature information.

3. The music generation method for a smart wearable device according to claim 1, characterized in that: Step S2 includes: Acquire historical music generation data, extract music features of the historical music generation data, and analyze historical performance patterns of users corresponding to the historical music generation data; Training a performance pattern recognition model according to the music features of the historical music generation data and the historical performance patterns of the corresponding user, so as to recognize the real-time performance pattern of the user in real time according to the performance pattern recognition model; A music control signal is generated according to the real-time performance mode of the user and the action feature information.

4. The music generation method for a smart wearable device according to claim 3, characterized in that: Generating a music control signal according to the real-time performance mode and the action feature information of the user includes: Determine the rhythm of the music control signal by combining the real-time performance mode and the action frequency of the action characteristic information; Analyzing the user's emotional characteristics according to the strength information of the action characteristic information, and determining the timbre of the music control signal according to the user's emotional characteristics and the real-time performance mode; The note of the music control signal is determined by identifying the gesture feature of the action feature information.

5. The music generation method for a smart wearable device according to claim 1, characterized in that: After step S3, the following steps are also included: Real-time action physiological data is captured, and the rhythm, volume and pitch of the action music are adjusted according to the emotional features and action features in the real-time action physiological data.

6. A music generating terminal for a smart wearable device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the following steps are implemented: S1. Acquire motion physiological data captured by at least two sensors of the smart wearable device, fuse the motion physiological data captured by each sensor, and obtain motion feature information; S2, converting the action feature information into a music control signal based on historical music generation data; S3. Mapping the music control signal to music elements of a preset music library, and generating action music corresponding to the captured action physiological data according to the mapped music elements.

7. The music generating terminal of a smart wearable device according to claim 6, characterized in that: In step S1, the motion physiological data captured by each sensor are fused to obtain motion feature information, including: Kalman filter, particle filter and extended Kalman filter are used to estimate the state of the action physiological data captured by each sensor; The state estimation results are weighted averaged or optimally fused to obtain the motion feature information.

8. The music generating terminal of a smart wearable device according to claim 6, characterized in that: Step S2 includes: Acquire historical music generation data, extract music features of the historical music generation data, and analyze historical performance patterns of users corresponding to the historical music generation data; Training a performance pattern recognition model according to the music features of the historical music generation data and the historical performance patterns of the corresponding user, so as to recognize the real-time performance pattern of the user in real time according to the performance pattern recognition model; A music control signal is generated according to the real-time performance mode of the user and the action feature information.

9. The music generating terminal of a smart wearable device according to claim 8, characterized in that: Generating a music control signal according to the real-time performance mode and the action feature information of the user includes: Determine the rhythm of the music control signal by combining the real-time performance mode and the action frequency of the action characteristic information; Analyzing the user's emotional characteristics according to the strength information of the action characteristic information, and determining the timbre of the music control signal according to the user's emotional characteristics and the real-time performance mode; The note of the music control signal is determined by identifying the gesture feature of the action feature information.

10. The music generating terminal of a smart wearable device according to claim 6, characterized in that: After step S3, the following steps are also included: Real-time action physiological data is captured, and the rhythm, volume and pitch of the action music are adjusted according to the emotional features and action features in the real-time action physiological data.

Images