Music generation and gait analysis method and system based on human body motion information

Abstract

Disclosed in the present invention are a music generation and gait analysis method and system based on human body motion information. The method comprises: acquiring human body motion information collected by an inertial sensor mounted on a human body; processing the human body motion information to obtain target human body motion information; on the basis of the target human body motion information and user input information, generating a music instruction; sending the music instruction to an electronic music player, and controlling the electronic music player to generate and play music on the basis of the music instruction; and on the basis of the music played by the electronic music player, performing gait analysis to obtain a gait analysis result. In the present invention, human body motion information is collected by means of a wearable inertial sensor, and personalized and instant music is generated on the basis of the collected human body motion information, so as to implement a mapping between human body actions and music; human body gait differences are reflected by means of the generated music, so as to implement gait analysis; and the present invention be used as a rehabilitation treatment tool to enhance sensorimotor synchronization and improve patient motivation for treatment.

Description

A method and system for music generation and gait analysis based on human motion information Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to a method, system, terminal, and computer-readable storage medium for music generation and gait analysis based on human motion information. Background Technology

[0002] In recent years, music therapy has become an important application in neuroscience and rehabilitation. Using music as feedback can help patients improve motor coordination, and it can also serve as a positive stimulus and motivation, encouraging patients to participate more actively in the rehabilitation process.

[0003] When generating music using traditional electronic music generators (electronic musical instruments), such as keyboards, drum machines, etc., the composer needs to consciously perform striking and other control actions. This requires the composer to undergo a certain level of training to generate meaningful music; otherwise, music generated by controlling these electronic instruments is often random and hardly worthy of being called a piece of music. Besides the methods mentioned above, existing artificial intelligence models can also generate music, such as the Suno AI music generator. However, intelligent music generators lack active human involvement in music creation, and therefore typically lack emotion, originality, and artistic quality.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a method, system, terminal, and computer-readable storage medium for music generation and gait analysis based on human motion information. This aims to solve the problems of existing music generation schemes using electronic musical instruments, which require experienced creators and have high operational requirements, and music generation schemes using artificial intelligence models, which lack individuality and thus result in poor effects.

[0006] To achieve the above-mentioned objectives, this invention provides a method for music generation and gait analysis based on human motion information, wherein the method includes:

[0007] Acquire human motion information collected by inertial sensors installed on the human body;

[0008] The human motion information is processed to obtain the target human motion information;

[0009] Based on the target human motion information and user input information, music commands are generated;

[0010] The music command is sent to the electronic music player, which then generates and plays music according to the music command.

[0011] Gait analysis was performed based on the music played by the electronic music player to obtain gait analysis results.

[0012] Optionally, acquiring the human motion information collected by the inertial sensor installed on the human body specifically includes:

[0013] The inertial sensor receives human motion information collected by the inertial sensor via the UDP communication protocol. The human motion information is acceleration data in the Z-axis direction. The inertial sensor is installed on any part of the human body.

[0014] Optionally, the target human motion information includes the acceleration and value in the Z-axis direction;

[0015] The process of processing the human motion information to obtain the target human motion information specifically involves:

[0016] The acceleration data in the Z-axis direction is accumulated for a set number of frames and then summed to obtain the sum of acceleration values ​​in the Z-axis direction.

[0017] Optionally, the music commands include Note On commands;

[0018] The step of generating music commands based on the target human motion information and user input information specifically includes:

[0019] Based on the Note On instruction in the MIDI protocol, a string structure is defined, which includes a data frame start marker, a data frame end marker, a data segmentation marker, and control bytes.

[0020] The system receives a string and parses the string. If the data parsing is correct, the user input information and the acceleration and value in the Z-axis direction are assigned to the corresponding positions of the control bytes in the string. If the data parsing is incorrect, the system receives the string again.

[0021] Extract the control information contained in the assigned string, and generate a Note On instruction based on the control information.

[0022] Optionally, the control bytes of the string include a first byte, a second byte, and a third byte;

[0023] Assigning the user input information and the acceleration and value in the Z-axis direction to the corresponding positions of the control bytes of the string specifically includes:

[0024] The number of playback channels in the user input information is assigned to the first byte of the string, which indicates that music is played on the playback channel corresponding to the number of playback channels.

[0025] The acceleration and value in the Z-axis direction are assigned to the second byte of the string to indicate that music is played using the acceleration and value in the Z-axis direction as the pitch.

[0026] The loudness value from the user input information is assigned to the third byte of the string to indicate that the music is played using the loudness value as the volume.

[0027] Optionally, sending the music command to the electronic music player and controlling the electronic music player to generate and play music according to the music command specifically includes:

[0028] The Note On command is sent to the electronic music player with a built-in audio codec via serial communication.

[0029] The audio codec of the electronic music player is controlled to decode the Note On command and generate a playable music melody;

[0030] The electronic music player is controlled to play the music melody through its speaker component.

[0031] Optionally, the step of performing gait analysis based on the music played by the electronic music player to obtain gait analysis results specifically includes:

[0032] The gait is determined to be normal or abnormal based on the music played by the electronic music player, and the gait analysis result is obtained.

[0033] If the gait analysis results indicate that the current gait is a normal gait, no warning will be issued;

[0034] If the gait analysis results indicate that the current gait is an abnormal gait, an alert will be issued.

[0035] To achieve the above-mentioned objectives, the present invention also provides a music generation and gait analysis system based on human motion information, the music generation and gait analysis system based on human motion information comprising:

[0036] Data acquisition module: used to acquire human motion information collected by inertial sensors installed on the human body;

[0037] Data processing module: used to process the human motion information to obtain target human motion information;

[0038] Music instruction generation module: used to generate music instructions based on the target human motion information and user input information;

[0039] Music generation module: used to send the music instructions to the electronic music player, and control the electronic music player to generate and play music according to the music instructions;

[0040] Gait analysis module: used to perform gait analysis based on the music played by the electronic music player and obtain gait analysis results.

[0041] To achieve the above-mentioned objectives, the present invention also provides a terminal, the terminal comprising: a memory, a processor, and a music generation and gait analysis program based on human motion information stored in the memory and executable on the processor, wherein when the music generation and gait analysis program based on human motion information is executed by the processor, it implements the steps of the music generation and gait analysis method based on human motion information as described above.

[0042] To achieve the above-mentioned objectives, the present invention also provides a computer-readable storage medium storing a music generation and gait analysis program based on human motion information. When the music generation and gait analysis program based on human motion information is executed by a processor, it implements the steps of the music generation and gait analysis method based on human motion information as described above.

[0043] This invention acquires human motion information collected by an inertial sensor mounted on the human body; processes the human motion information to obtain target human motion information; generates music commands based on the target human motion information and user input; sends the music commands to an electronic music player, controlling the electronic music player to generate and play music according to the music commands; and performs gait analysis based on the music played by the electronic music player to obtain gait analysis results. This invention collects human motion information using a wearable inertial sensor, generates personalized and real-time music based on the collected human motion information, and realizes the mapping between human movement and music. It provides a more interactive and interesting music generation mode based on human motion, without requiring creators to have music composition experience; it reflects differences in human gait through the generated music, achieving gait analysis; and it can be used as a rehabilitation tool to enhance sensorimotor synchronization and improve patients' treatment motivation, especially for patients with motor dysfunction or gait problems, promoting neurological rehabilitation by enhancing sensorimotor synchronization and increasing participation in rehabilitation. Attached Figure Description

[0044] Figure 1 is a structural diagram of an existing electronic music generation system;

[0045] Figure 2 is a flowchart of a preferred embodiment of the music generation and gait analysis method based on human motion information of the present invention;

[0046] Figure 3 is a flowchart illustrating the working principle of the core controller of this invention;

[0047] Figure 4 is a schematic diagram of the string structure of the present invention;

[0048] Figure 5 is a structural diagram of the wearable music generation system of the present invention;

[0049] Figure 6 is a waveform diagram of the sound signal obtained in the experiment of this invention;

[0050] Figure 7 is a spectral diagram of the sound signal obtained in the experiment of this invention;

[0051] Figure 8 is a spectrum diagram of the sound signal obtained in the experiment of this invention;

[0052] Figure 9 is a structural diagram of a preferred embodiment of the music generation and gait analysis system based on human motion information of the present invention.

[0053] Figure 10 is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0055] In recent years, music therapy has become an important application in neuroscience and rehabilitation. Using music as feedback can help patients improve motor coordination, and it can also serve as a positive stimulus and motivation, encouraging patients to participate more actively in the rehabilitation process.

[0056] Beyond traditional instruments, electronically generated music has flourished in recent decades. Electronic music is music created and performed using electronic instruments, synthesizers, computers, and digital technologies. It is characterized by the use of electronic devices and software to synthesize, process, and modify sounds to create unique musical effects and sonic textures. The origins of electronic music can be traced back to the early 20th century, when musicians began using electronic instruments and devices to explore new ways of composing music. With continuous technological advancements, electronic music has gradually become an independent genre, experiencing widespread development and popularity in the latter half of the 20th century and the early 21st century.

[0057] Currently, the creation of electronic music requires the human creation of musicians, who organize different musical elements into melodies or chords. For example, drum machines, percussion instrument sampling, or programmed drum beats are used to define musical rhythms; synthesizers, samplers, and audio effects are used to adjust and process sounds to create unique timbres; and the arrangement, transitions, and structure of musical elements such as tracks are determined by the musician's personalized composition. Existing electronic music generation systems, as shown in Figure 1(a), first require an electronic music generator 1 (also called an electronic instrument or MIDI instrument) to generate an "electronic music input signal"; the generated signal is converted into a MIDI signal (also called MIDI instructions or MIDI data) by a built-in digital audio encoder; then the MIDI signal is sent to an electronic music player 2 with a built-in digital audio decoder, ultimately generating a playable melody. As shown in Figure 1(b), some of these steps can be completed by connecting a computer 3 with a built-in digital audio codec, and the computer 3 is connected to the electronic music player 2 via a MIDI interface device 4.

[0058] Musical Instrument Digital Interface (MIDI) is a standard electronic communication protocol for generating electronic music. The MIDI protocol defines various note codes or playing codes for electronic musical instruments and other performance devices (such as synthesizers), allowing electronic instruments, computers, mobile phones, or other stage performance equipment to connect, adjust, and synchronize with each other, enabling the real-time exchange of performance data.

[0059] In existing technologies, when generating music using traditional electronic music generators (electronic musical instruments), such as electronic keyboards, drum machines, etc., the composer needs to consciously perform striking and other control actions. This requires the composer to undergo a certain amount of training to generate meaningful music; otherwise, music generated by controlling these electronic instruments is often random and can hardly be called a piece of music. Besides the methods mentioned above, existing artificial intelligence models can also generate music, such as the SunoAI music generator. However, intelligent music generators lack active human involvement in music creation, and therefore usually lack emotion, originality, and artistic quality.

[0060] To address the aforementioned technical problems, this invention provides a method for music generation and gait analysis based on human motion information. The method involves acquiring human motion information collected by an inertial sensor mounted on the human body; processing the human motion information to obtain target human motion information; generating music commands based on the target human motion information and user input; sending the music commands to an electronic music player to control the player to generate and play music according to the commands; and performing gait analysis based on the music played by the player to obtain gait analysis results. This invention collects human motion information using wearable inertial sensors and generates personalized, real-time music based on this information, achieving a mapping between human movement and music. By combining inertial sensors and a music generation system, it provides a more interactive and engaging music generation mode based on human motion, without requiring creators to have prior music composition experience. The generated music reflects differences in human gait, enabling gait analysis. It can be used as a rehabilitation tool to enhance sensorimotor synchronicity and improve patient motivation, especially for patients with motor dysfunction or gait problems, promoting neurological rehabilitation by enhancing sensorimotor synchronicity and increasing participation in rehabilitation.

[0061] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0062] A preferred embodiment of the music generation and gait analysis method based on human motion information of the present invention, as shown in Figure 2, specifically includes:

[0063] S1. Acquire human motion information collected by inertial sensors installed on the human body.

[0064] In one implementation of this embodiment, acquiring human motion information collected by an inertial sensor mounted on the human body specifically involves:

[0065] Human motion information is received from an inertial sensor via the UDP communication protocol. The human motion information is acceleration data in the Z-axis direction. The inertial sensor is installed on any part of the human body.

[0066] Specifically, the core components of the inertial sensor developed in this invention are a 9-axis inertial sensor chip (BNO055) and a microcontroller (esp32_wroom). The 9-axis inertial sensor chip (BNO055) is used to collect human motion information. This invention acquires the Z-axis acceleration data collected by the BNO055 chip, with a sampling rate of 100Hz. The microcontroller is used to send the data collected by the 9-axis inertial sensor chip to the core controller via the UDP communication protocol.

[0067] S2. Process the human motion information to obtain the target human motion information.

[0068] In one implementation of this embodiment, the target human motion information includes the acceleration and value in the Z-axis direction;

[0069] The human motion information is processed to obtain the target human motion information, specifically:

[0070] Acceleration data in the Z-axis direction is accumulated for a set number of frames (5 frames) and then summed to obtain the sum of acceleration values ​​in the Z-axis direction.

[0071] Specifically, the core controller employed in this invention includes a Raspberry Pi-based embedded system for receiving inertial sensor data and deploying a mapping algorithm; a VS1053b-based digital audio codec for generating music instructions (MIDI instructions); and a serial connector (CH340) for connecting the core controller to an electronic music player. After receiving inertial sensor data via the UDP communication protocol, the embedded system of the core controller processes the inertial sensor data using the proposed "motion-music mapping algorithm," then encodes the processed inertial sensor data into MIDI instructions using the digital audio codec, and finally sends the MIDI instructions to the electronic music player via serial communication.

[0072] In this invention, the embedded system, i.e., the microcontroller, in the core controller receives inertial sensor data, namely the aforementioned human motion information, specifically the acceleration along the Z-axis. After accumulating five frames of data, the sum is obtained to obtain the sum of acceleration values ​​(AccZ) along the Z-axis, which is then used as the input value for the mapping algorithm. This invention chooses to sum five frames of data to increase the degree of variation in adjacent inertial sensor signals. The variation in adjacent AccZ is greater than the original variation in adjacent acceleration values, resulting in a more noticeable change in sound after mapping. Furthermore, choosing five frames of data avoids excessive delay.

[0073] S3. Generate music commands based on the target human motion information and user input information.

[0074] In one implementation of this embodiment, the music command includes the Note On command;

[0075] Based on the target human motion information and user input information, music commands are generated, specifically including:

[0076] S31. Define a string structure according to the Note On instruction in the MIDI protocol. The string structure includes a data frame start marker, a data frame end marker, a data segmentation marker, and control bytes.

[0077] S32. Receive the string and parse the string. If the data parsing is correct, assign the user input information and the acceleration and value in the Z-axis direction to the corresponding positions of the control bytes of the string. If the data parsing is incorrect, receive the string again.

[0078] S33. Extract the control information contained in the assigned string, and generate the Note On instruction based on the control information.

[0079] In one implementation of this embodiment, the control bytes of the string include a first byte, a second byte, and a third byte;

[0080] Assigning user input information and the acceleration value along the Z-axis to the corresponding positions in the control bytes of the string, specifically including:

[0081] S321. Assign the number of playback channels in the user input information to the first byte of the string to indicate that music will be played on the playback channel corresponding to the number of playback channels.

[0082] S322. Assign the acceleration and value in the Z-axis direction to the second byte of the string to indicate that the music is played with the acceleration and value in the Z-axis direction as the pitch.

[0083] S323. Assign the loudness value from the user input information to the third byte of the string to indicate that the loudness value is used as the volume for playing music.

[0084] Specifically, embedded code is deployed in the core controller. After the core controller is powered on, the functions implemented by this code are shown in Figure 3, which illustrates the working process of the core controller, including:

[0085] 1) The embedded system in the core controller, i.e., the microcontroller, receives data from the inertial sensor, accumulates 5 frames of data, and sums them to obtain the acceleration sum value AccZ in the Z-axis direction. AccZ is then used as the input value for the mapping algorithm. Here, choosing to sum 5 frames of data increases the degree of change in adjacent inertial sensor signals. The change in adjacent AccZ is greater than the original change in adjacent acceleration values, which can lead to a more obvious change in sound after mapping. On the other hand, choosing 5 frames of data will not cause excessive delay.

[0086] 2) Based on the music instruction in the MIDI protocol, namely the MIDI instruction - Note On instruction, define a new string structure, as shown in Figure 4. This string structure includes a data frame start marker (#), a data frame end marker (*), two data separation markers ( / ), and three control bytes (the first byte, the second byte, and the third byte, represented as data1, data2, and data3, respectively). The three control bytes data1, data2, and data3 correspond to the three data bytes of the "Note On instruction," and their functions are playback channel, pitch, and loudness, respectively.

[0087] 3) This string is sent to the embedded system of the core controller, i.e., the microcontroller, via serial communication for data parsing. The start marker (#) and end marker (*) of the data frame are used to determine whether the data frame was successfully received. If the marker is not recognized, it will be judged as "data parsing error (data number does not match)" and the data will be received again; if the marker is successfully recognized, it will be judged as "data parsing is correct".

[0088] 4) If the data parsing is correct, assign values ​​to the three control bytes of the string, including:

[0089] i. Assign the value "0x90" to the data1 control byte position of this string, indicating that the Note on instruction (play a note) is executed on channel 1;

[0090] ii. Assign the value "AccZ" to the data2 control byte position of this string as a signal to control the pitch;

[0091] iii. Assign the value "90" to the data3 control byte position of this string to indicate the dynamics (volume) of the note;

[0092] 5) The core controller's audio codec substitutes the assigned string as three data bytes into the "Note On instruction" to generate a music instruction, i.e., a MIDI instruction. This instruction is then sent to the electronic music player's built-in audio codec via serial communication. The electronic music player's audio codec decodes the instruction to generate a musical melody, which is then transmitted to the electronic music player via an audio cable to produce music.

[0093] S4. Send the music command to the electronic music player, and control the electronic music player to generate and play music according to the music command.

[0094] In one implementation of this embodiment, a music command is sent to an electronic music player, which then generates and plays music according to the command. Specifically, this includes:

[0095] S41. Send the Note On command to the electronic music player with built-in audio codec via serial communication.

[0096] S42. Control the audio codec of the electronic music player to decode the Note On command and generate a playable music melody.

[0097] S43. Control the electronic music player to play music melodies through the speaker component of the electronic music player.

[0098] Specifically, the core controller connects to the electronic music player via a serial connector (CH340) and sends music instructions (i.e., MIDI instructions) - Note On instructions to the electronic music player with a built-in audio codec via serial communication. The electronic music player's audio codec decodes the music instructions to generate a playable music melody, which is then played by the electronic music player through an audio cable. The electronic music player then plays the music through its speaker.

[0099] S5. Perform gait analysis based on the music played by the electronic music player to obtain the gait analysis results.

[0100] In one implementation of this embodiment, gait analysis is performed based on the music played by the electronic music player to obtain gait analysis results, specifically including:

[0101] The gait is determined to be normal or abnormal based on the music played by the electronic music player, and the gait analysis results are obtained.

[0102] If the gait analysis results indicate that the current gait is a normal gait, no warning will be issued;

[0103] If the gait analysis results indicate that the current gait is an abnormal gait, an alert will be issued.

[0104] In recent years, music therapy has become an important application in neuroscience and rehabilitation. Using music as feedback information can help patients improve motor coordination and also serve as a positive stimulus and motivation, encouraging patients to participate more actively in the rehabilitation process. This invention achieves a mapping between human movement and music, generating music based on human movement information. Therefore, music can reflect human movement; for example, it can use music signals to reflect the differences between normal and abnormal gait, enabling gait analysis.

[0105] In addition, music generation and gait analysis methods based on human motion information also include:

[0106] Collect historical human motion information from inertial sensors installed on the human body;

[0107] Historical human motion information is processed and calibrated to obtain a dataset;

[0108] The dataset is divided into training and testing sets. The deep learning network model is trained and tested using the training and testing sets to obtain a music generation and gait analysis model.

[0109] The system acquires real-time human motion information collected by inertial sensors, inputs this information into a music generation and gait analysis model, and outputs the music generation and gait analysis results.

[0110] This invention can also use artificial intelligence and machine learning algorithms to analyze human motion data and generate music. This method uses deep learning models to understand motion patterns and convert them into musical output. Through deep learning, complex motion data is automatically analyzed and converted into music, reducing human intervention; it can create entirely new musical styles and patterns, possessing high innovative potential.

[0111] In summary, this invention innovatively proposes a music generation method based on human motion information, realizing a wearable music generation system that uses a core controller to generate music signals based on a wearable inertial measurement unit (IMU). By combining the aforementioned hardware system with music theory, an innovative "motion-music mapping algorithm" is proposed, which can generate corresponding sounds / music based on different human movements. The design route of the entire system is shown in Figure 5. The inertial sensor 4 is worn on different parts of the human body 5. The inertial sensor 4 collects human motion information and sends it to the core controller 6 via the UDP communication protocol. The core controller 6 generates corresponding music instructions, i.e., MIDI instructions, based on the human motion information, and then sends the MIDI instructions to the electronic music player 2 via serial communication. The electronic music player 2 generates and plays music based on the MIDI instructions.

[0112] This invention proposes a wearable music generation system based on human motion information (actions). The system consists of multiple wearable inertial measurement units (IMUs) and a digital audio codec. After the user wears the IMUs, the IMUs can acquire human motion information in real time during movement and wirelessly transmit it to the digital audio codec. Electronic music is generated via the MIDI protocol and played through external speakers, headphones, or a host computer with a sound card. In other words, this invention innovatively combines wearable inertial sensors with a music generation system to create a more interactive and engaging music generation mode, method, and system, providing personalized and immediate musical feedback. In terms of applications, this invention can provide a new rehabilitation tool, especially for patients with motor dysfunction or gait problems, promoting neurorehabilitation by enhancing sensorimotor synchronization and increasing participation in rehabilitation.

[0113] The method of this invention was verified through experiments. An IMU inertial sensor was attached to the upper of a shoe to acquire sensor signals at different walking speeds (including fast walking at approximately 3 m / s and slow walking at approximately 1.2 m / s). Then, different music was generated using the method proposed in this invention.

[0114] The experiment compared the differences in the generated music from multiple dimensions, such as waveform diagrams, spectrograms, and frequency spectra of the sound signals, thus proving that the method proposed in this invention can generate different sound signals based on different gaits. Therefore, it has application value and prospects in the fields of gait analysis and gait rehabilitation.

[0115] The waveforms of the sound signals obtained in the experiment are shown in Figure 6. Figure 6(a) shows the waveform of the sound signal corresponding to fast walking, and Figure 6(b) shows the waveform of the sound signal corresponding to normal walking. The waveforms reflect the temporal characteristics of the generated sound signals, visualizing the changes in amplitude (loudness) of the two sound signals over time. It can be seen that the waveforms of the sounds generated by a person walking at different speeds in this invention are significantly different, reflected in the shape and structure of the sound waveforms, indicating that the generated sounds have specific characteristics and can effectively reflect gait differences.

[0116] The spectra of the sound signals obtained in the experiment are shown in Figure 7. Figure 7(a) shows the spectrum of the sound signal corresponding to fast walking, and Figure 7(b) shows the spectrum of the sound signal corresponding to normal walking. The spectra reveal the frequency distribution in the sound signal. The colors in the spectra represent the intensity or power of the corresponding frequency components and reflect how the frequency content of the signal changes over time. This spectra shows obvious periodic patterns, indicating the presence of harmonics in the sound signal generated by walking. However, the forms of the sound spectra produced by different walking speeds are different, with the periodic changes being more obvious during fast walking. Since the spacing between adjacent spectra represents the harmonic relationship, this result may indicate that the sound signal generated during fast walking is more stable.

[0117] The spectrograms of the sound signals obtained in the experiment are shown in Figure 8. Figure 8(a) shows the spectrogram of the sound signal corresponding to fast walking, and Figure 8(b) shows the spectrogram of the sound signal corresponding to normal walking. The spectrogram provides information about the amplitude or power of each frequency component. The height or intensity of each peak in the spectrum represents the intensity or energy of that specific frequency component in the signal. The spectrogram allows us to gain a deeper understanding of the relative loudness or intensity of different frequencies in the sound, and which frequencies dominate or are prominent in the signal. The results of the spectrogram also illustrate the differences in frequency distribution of the sound produced by different walking speeds. These differences are of important guiding significance for further optimization of the pitch quality, timbre, or characteristic sounds of the generated audio.

[0118] Furthermore, based on the aforementioned method for music generation and gait analysis based on human motion information, this invention also provides a system for music generation and gait analysis based on human motion information. A preferred embodiment of this system, as shown in Figure 9, specifically includes:

[0119] Data acquisition module 01: Used to acquire human motion information collected by inertial sensors installed on the human body;

[0120] Data processing module 02: Used to process human motion information to obtain target human motion information;

[0121] Music instruction generation module 03: Used to generate music instructions based on the target human body movement information and user input information;

[0122] Music generation module 04: Used to send music commands to the electronic music player, controlling the electronic music player to generate and play music according to the music commands;

[0123] Gait Analysis Module 05: Used to perform gait analysis based on music played by an electronic music player and obtain gait analysis results.

[0124] Furthermore, based on the aforementioned method and system for music generation and gait analysis based on human motion information, this invention also provides a terminal. A preferred embodiment of the terminal, as shown in FIG10, specifically includes a processor 10, a memory 20, and a display 30. FIG10 only shows some components of the terminal; however, it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented alternatively.

[0125] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, or flash card. Furthermore, the memory 20 may include both internal and external storage units. The memory 20 is used to store application software and various types of data installed on the terminal, such as the terminal's program code. The memory 20 may also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a music generation and gait analysis program 40 based on human motion information. This music generation and gait analysis program 40 can be executed by the processor 10 to implement the steps of the music generation and gait analysis method based on human motion information in this application.

[0126] In some embodiments, processor 10 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 20 or process data, such as executing a music generation and gait analysis program 40 based on human motion information.

[0127] In some embodiments, display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0128] In one embodiment, when the processor 10 executes the music generation and gait analysis program 40 based on human motion information stored in the memory 20, it implements the steps of the music generation and gait analysis method based on human motion information as described above.

[0129] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a music generation and gait analysis program based on human motion information, and when the music generation and gait analysis program based on human motion information is executed by a processor, it implements the steps of the music generation and gait analysis method based on human motion information as described above.

[0130] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0131] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0132] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A music generation and gait analysis method based on human motion information, characterized in that, The music generation and gait analysis method based on human motion information includes: Acquire human motion information collected by inertial sensors installed on the human body; The human motion information is processed to obtain the target human motion information; Based on the target human motion information and user input information, music commands are generated; The music command is sent to the electronic music player, which then generates and plays music according to the music command. Gait analysis was performed based on the music played by the electronic music player to obtain gait analysis results.

2. The music generation and gait analysis method based on human motion information according to claim 1, characterized in that, The acquisition of human motion information collected by inertial sensors installed on the human body specifically includes: The inertial sensor receives human motion information collected by the inertial sensor via the UDP communication protocol. The human motion information is acceleration data in the Z-axis direction. The inertial sensor is installed on any part of the human body.

3. The music generation and gait analysis method based on human motion information according to claim 2, characterized in that, The target human motion information includes the acceleration and value in the Z-axis direction; The process of processing the human motion information to obtain the target human motion information specifically involves: The acceleration data in the Z-axis direction is accumulated for a set number of frames and then summed to obtain the sum of acceleration values ​​in the Z-axis direction.

4. The music generation and gait analysis method based on human motion information according to claim 3, characterized in that, The music commands include the Note On command; The step of generating music commands based on the target human motion information and user input information specifically includes: Based on the Note On instruction in the MIDI protocol, a string structure is defined, which includes a data frame start marker, a data frame end marker, a data segmentation marker, and control bytes. The system receives a string and parses the string. If the data parsing is correct, the user input information and the acceleration and value in the Z-axis direction are assigned to the corresponding positions of the control bytes in the string. If the data parsing is incorrect, the system receives the string again. Extract the control information contained in the assigned string, and generate a Note On instruction based on the control information.

5. The music generation and gait analysis method based on human motion information according to claim 4, characterized in that, The control bytes of the string include the first byte, the second byte, and the third byte; Assigning the user input information and the acceleration and value in the Z-axis direction to the corresponding positions of the control bytes of the string specifically includes: The number of playback channels in the user input information is assigned to the first byte of the string, which indicates that music is played on the playback channel corresponding to the number of playback channels. The acceleration and value in the Z-axis direction are assigned to the second byte of the string to indicate that music is played using the acceleration and value in the Z-axis direction as the pitch. The loudness value from the user input information is assigned to the third byte of the string to indicate that the music is played using the loudness value as the volume.

6. The music generation and gait analysis method based on human motion information according to claim 4 or 5, characterized in that, The step of sending the music command to the electronic music player and controlling the electronic music player to generate and play music according to the music command specifically includes: The Note On command is sent to the electronic music player with a built-in audio codec via serial communication. The audio codec of the electronic music player is controlled to decode the Note On command and generate a playable music melody; The electronic music player is controlled to play the music melody through its speaker component.

7. The method for music generation and gait analysis based on human motion information according to claim 1, characterized in that, The step of performing gait analysis based on the music played by the electronic music player to obtain gait analysis results specifically includes: The gait is determined to be normal or abnormal based on the music played by the electronic music player, and the gait analysis result is obtained. If the gait analysis results indicate that the current gait is a normal gait, no warning will be issued; If the gait analysis results indicate that the current gait is an abnormal gait, an alert will be issued.

8. A music generation and gait analysis system based on human motion information, characterized in that, The music generation and gait analysis system based on human motion information includes: Data acquisition module: used to acquire human motion information collected by inertial sensors installed on the human body; Data processing module: used to process the human motion information to obtain target human motion information; Music instruction generation module: used to generate music instructions based on the target human motion information and user input information; Music generation module: used to send the music instructions to the electronic music player, and control the electronic music player to generate and play music according to the music instructions; Gait analysis module: used to perform gait analysis based on the music played by the electronic music player and obtain gait analysis results.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a music generation and gait analysis program based on human motion information stored in the memory and executable on the processor. When the music generation and gait analysis program based on human motion information is executed by the processor, it implements the steps of the music generation and gait analysis method based on human motion information as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a music generation and gait analysis program based on human motion information. When the music generation and gait analysis program based on human motion information is executed by a processor, it implements the steps of the music generation and gait analysis method based on human motion information as described in any one of claims 1-7.

Images