Apparatus for generating artificial intelligence (AI) music based on gait

Abstract

To provide an apparatus for generating AI music based on a movement and gait of a user.SOLUTION: Provided is a system configuration 100 including: a sensor assembly 102 provided on a foot of a user, the sensor assembly including a pressure sensing device and an inertial sensing module and being configured to collect data related to a movement of the user; a computing electronic device 104 communicatively connected to the sensor assembly and being configured to receive the data, analyze the data related to the movement by a gait analysis module to obtain gait parameters of the user and then extract rhythm parameters related to the movement among the gait parameters of the user by a rhythm parameter extraction module; and a cloud server 106 having an AI music generation module 360 communicatively connected to the electronic device and being configured to receive the rhythm parameters related to the movement of the user, convert the rhythm parameters into a corresponding beat number, pitch of sound, and weight of sound, and add a melody, thereby generating and outputting music.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to the technical field of music generation using artificial intelligence (AI), and in particular to an AI music generation device based on gait. [Background technology]

[0002] With the development of electronics and information and communication technologies, various sensors have become capable of detecting and extracting information about a user's movements and gait. In the prior art, for example, Patent Document 1 attempts to develop an electronic tap shoe that detects a dancer's movements using pressure sensors or other methods and then transmits corresponding Musical Instrument Digital Interface (MIDI) notes via a cable or wirelessly. For example, this patent provides a shoe-activated synthesizer device that can convert shoe movements into audible sounds. The synthesizer device includes a shoe with at least one trigger member that can generate a trigger signal when the shoe bends to a predetermined degree. The shoe bends when it contacts the floor. Thus, by contacting different parts of the shoe with the floor in a controlled manner, a person can selectively control the generation of a trigger signal from any trigger member contained within the shoe. Also provided is a synthesizer circuit coupled to each trigger member contained within the shoe. Upon receiving a trigger signal from the shoe, the synthesizer circuit generates an audible sound through a speaker.

[0003] Pressure sensors may be embedded in floors or attached to the surface of floors to form home game controllers or devices for tracking user movement and activity to follow fitness and exercise progress.These sensors can transmit data packets to a receiver that connects directly to a mobile device.

[0004] Music and dance are admired hobbies and entertaining activities in modern society, and learning to dance has become one of the most popular activities around the world. However, learning very fast and highly dynamic movements synchronized with your favorite music can be challenging.

[0005] For example, when performing coordinated movements in time and space, such as dancing or exercise, the movement of the user's body becomes a complex process.

[0006] Rhythm is becoming increasingly closely related to people's daily lives. For example, people can exercise in sync with the rhythm of an audio signal, design rhythm games based on the rhythm of an audio signal, or design flashing lights in sync with the rhythm of an audio signal. Generally, to obtain the rhythm of an audio signal, it is necessary to identify rhythm points from the audio signal. Therefore, how to accurately identify rhythm points in an audio signal is currently an urgent problem in the relevant technical fields.

[0007] Automatically generating music is not easy, because many factors must be considered, including not only melody generation but also accompaniment generation, lyrics generation, instrumentation, and arrangement. However, with the rapid development of artificial intelligence technology, it is now possible to use generative AI technology (including machine learning models and deep learning models) to collect the rhythm of a user's movements and footsteps and generate music. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] U.S. Patent No. 5,765,300 Summary of the Invention [Means for solving the problem]

[0009] To achieve the above object, the present invention provides a gait-based AI music generation device, including: a sensor assembly provided on a user's foot, the sensor assembly including a pressure sensing device and an inertial sensing module, and configured to collect data related to the user's motion; a computing electronic device communicatively connected to the sensor assembly, the computing electronic device configured to receive the data related to the user's motion, analyze the data related to the motion using a gait analysis module provided in the computing electronic device to obtain gait parameters of the user, and then extract rhythmic parameters related to the user's motion from the gait parameters using a rhythmic parameter extraction module provided in the computing electronic device; and an AI music generation module communicatively connected to the computing electronic device, configured to receive the rhythmic parameters related to the user's motion, convert the rhythmic parameters related to the motion into corresponding beats, pitch, and weight, and add a melody using an AI algorithm to generate and output music.

[0010] In one embodiment, the gait-based AI music generation device further includes a music-gait interaction module provided in the computing electronic device, which generates a moving image in accordance with the rhythm of the music generated by the AI ​​music generation module and is used for interaction with the user.

[0011] In one embodiment, the pressure sensing device includes a plurality of pressure sensors provided in the toe region and heel region of adjacent feet of the user and used to detect pressure distribution in the toe region and heel region, respectively.

[0012] In one embodiment, the inertial sensing module includes a hybrid accelerometer-gyroscope sensor, which is mounted on the arches of adjacent feet of the user and is used to detect the motion of the user's feet.

[0013] In one embodiment, the inertial sensing module further includes a GPS sensor.

[0014] In one embodiment, the gait parameters include the user's strike times, stride length, step length, pitch, stand time, forward step, backward step, jump, and movement trajectory. The movement rhythm parameters include cadence, speed, jump height, force magnitude, and stride length.

[0015] In one embodiment, the sensor assembly further includes a microprocessor used for collecting and analyzing electronic signals detected by the pressure sensing device, the inertial sensing module, and the GPS sensor and converting them into corresponding foot pressure distribution information, acceleration information, and GPS information; a memory connected to the microprocessor and used for storing the foot pressure distribution information, the acceleration information, and the GPS information; a wireless transceiver connected to the microprocessor and used for wirelessly transmitting the foot pressure distribution information, the acceleration information, and the GPS information to an external electronic device; and a power supply used for supplying electricity to the pressure sensing device, the inertial sensing module, the GPS sensor, the microprocessor, the memory, and the wireless transceiver.

[0016] In one embodiment, the wireless transceiver is a Bluetooth chip (Bluetooth is a registered trademark), WiFi or similar data transmission / reception device.

[0017] In one embodiment, the AI ​​music generation module is provided on a cloud server, and the cloud server is a general server or a generation AI server. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 shows a system configuration 100 for implementing an AI music generation device based on gait. [Figure 2] FIG. 2 shows a schematic diagram of the distribution and wiring of pressure sensors provided in accordance with the present invention. [Figure 3] FIG. 3 shows a schematic diagram of the functional blocks of a sensor assembly provided in accordance with an embodiment of the present invention. [Figure 4] FIG. 4 shows a schematic system block diagram of the sensor module and its communication with an external computing electronic device. [Figure 5] FIG. 5 shows a functional block diagram corresponding to the system configuration of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0019] Herein, the present invention will be described in detail with respect to specific embodiments and aspects thereof. It should be noted that these descriptions are intended to interpret and explain the structure or step flow of the present invention, and are not intended to limit the scope of the claims of the present invention. Therefore, in addition to the specific and preferred embodiments described in the specification, the present invention can be broadly implemented in other different embodiments. Below, embodiments of the present invention will be described using specific specific embodiments. Those skilled in the art can easily understand the effects and advantages of the present invention from the contents disclosed herein. The present invention can also be operated and practiced using other specific embodiments. The details described in this specification can be applied according to different needs, and various supplements or modifications may be made without departing from the spirit of the present invention.

[0020] The present invention provides an AI music generation device based on gait. The concept of this device is to use an AI algorithm to compose music based on information about a user's movement and gait. For example, in a badminton match, players have numerous parameters related to their gait, such as jumps, forward motion, and backward motion. From these parameters, related parameters such as rhythm, speed, jump height, force, and step radius can be extracted. These parameters can correspond to beat rate, pitch, weight, etc., so by adding a melody using an AI algorithm, AI music related to exercise can be generated.

[0021] Based on the above concept, reference is made to FIG. 1 , which illustrates a system configuration 100 for realizing the above-described gait-based AI music generation device. The system configuration 100 includes a sensor assembly 102, a computing electronic device 104, and a cloud server 106. The sensor assembly 102 is communicatively connected to the external computing electronic device 104. The external computing electronic device 104 is capable of receiving data collected by the sensor assembly 102. The computing electronic device 104 is also connectable to the cloud server 106 for communication via the Internet. According to an embodiment of the present invention, the cloud server 106 may be a general network server or a generation AI server.

[0022] According to an embodiment of the present invention, the sensor assembly 102 includes a wireless transceiver configured to wirelessly transmit data collected by the sensor assembly 102 regarding the user's 108 movements to a signal receiver of the external computing electronic device 104. As discussed in more detail below, the sensor assembly 102 is mountable on the body of the user 108 (e.g., on the shoe or foot of the user 108) and includes a plurality of pressure sensors and an inertial sensing module (including a wireless transceiver). According to an embodiment of the present invention, the plurality of pressure sensors and the inertial sensing module (including a wireless transceiver) may be separate or integrated together. According to an embodiment of the present invention, the sensor assembly 102 is mountable on both feet of the user. Signals generated by the sensor assembly 102 and transmitted from the wireless transceivers included in the sensor assembly 102 (including the data regarding the user's 108 movements described above) can be received by a wireless receiver (not shown) located on the external computing electronic device 104. The computing electronic device 104 may include, for example, a computing device such as a smartphone, a laptop, a tablet PC, or a personal computer.

[0023] According to an embodiment of the present invention, digital data related to the user's 108 movements (e.g., data related to gait, such as jumps, steps, and steps) is wirelessly transmitted to the computing electronic device 104, enabling gait analysis. Gait parameters are then acquired and uploaded to the cloud server 106 via a network for subsequent analysis and music generation. The system further includes an application program installed on the computing electronic device 104. The application program includes instructions for transmitting and receiving data between the sensor assembly 102, the computing electronic device 104 (e.g., a mobile device such as a smartphone or tablet PC), and the cloud server 106. The application program may be based on an Android, Windows 10, or iOS platform and may upload signals to the cloud server 106 for storage and / or processing. In the system configuration 100, data related to the user's 108 movements can be continuously collected by the computing electronic device 104. The executable program is executed by the computing electronic device 104 to, for example, execute algorithms related to gait analysis to extract gait parameters from the received user's movements.

[0024] To explain the sensor assembly 102 in detail, as shown in FIG. 2, the sensor assembly 102 includes a plurality of pressure sensors and an inertial sensing module. The plurality of pressure sensors and the inertial sensing module may be integrated together or may be separate. For convenience of explanation, FIG. 2 only shows an example in which the plurality of pressure sensors and the inertial sensing module in the sensor assembly 102 are integrated together. For example, they are integrated into an insole 210. However, examples of the sensor assembly 102 that do not deviate from the spirit of the present invention, such as being integrated into a shoe or sole, also fall within the scope of protection of the present invention. Referring to FIG. 2, pressure sensors (not shown) may be provided only in the toe region 215, the lateral arch region 219, and the heel region 220 of the entire insole. The toe region 215 corresponds to the toe region and the forefoot region. Each of the above regions may have a plurality of pressure sensors (not shown), and pressure sensors (not shown) of different densities may be provided as needed. Each pressure sensor is electrically connected to the inertial sensing module 216 by a lead 222. In one embodiment, the pressure sensors are capacitive pressure sensors. A plurality of capacitive pressure sensors and corresponding wires 224 form a flexible pressure sensing device. These sensors are mounted in different locations on the insole 210 to detect foot pressure distribution in different regions of the user's foot (e.g., the aforementioned toe region 215, lateral arch region 219, and heel region 220). However, in alternative embodiments, the pressure sensors may be resistive pressure sensors. The inertial sensing module 216 also includes a wireless transceiver, such as a Bluetooth chip (Bluetooth is a registered trademark), and other electronic components (including an inertial measurement unit and a GPS sensor), all of which are built into the arch pad 214 in the arch portion of the insole 210. In one embodiment, the inertial sensing module 216 may be an electronic sensing module integrated on a printed circuit board and have connection terminals electrically connected to the plurality of pressure sensing devices.

[0025] According to embodiments of the present invention, the sensor assembly 102 can provide data / information regarding a user's physical movements. In some embodiments, the inertial sensing module 216 of the sensor assembly 102 can include: (1) a wearable, wireless, real-time motion detection device or IMU (inertial measurement units); or (2) a wearable, wireless, real-time, multi-area plantar pressure / 6-dimensional motion capture (IMU) combined device.

[0026] According to an embodiment of the present invention, the inertial sensing module 216 includes at least devices such as an accelerometer, a gyroscope, and a GPS sensor. The accelerometer and gyroscope may be sensors fabricated based on Micro-Electro-Mechanical Systems (MEMS) technology. The accelerometer and gyroscope may be integrated into a combined sensor having a six-axis accelerometer and gyroscope with six degrees of freedom.

[0027] Thus, the exemplary sensor assembly 102 may be a combined multi-region plantar pressure / six-degrees-of-freedom motion detection device. During a user's movement, the sensor assembly 102 records the user's foot pressure (via multiple pressure sensors) and six-degrees-of-freedom motion distribution (via the inertial sensing module 216). In some embodiments, the foot pressure / six-degrees-of-freedom motion detection device has a variable recording duration interval. Additionally, for the foot pressure distribution, which includes the toe region 215, the lateral arch region 219, and the heel region 220, a sampling circuit acquires pressure point data for each foot to generate a foot pressure chart for each sampling interval.

[0028] The sensor assembly 102 may include an inertial sensing module 216 that uses a combined sensor with a six-axis accelerometer and a gyroscope to form a six-dimensional motion detection device (also referred to as a six-axis inertial measurement unit). The device detects changes in motion using a six-degree-of-freedom Micro-Electro-Mechanical Systems (MEMS)-based sensor. The six-dimensional motion detection device detects changes in motion by detecting linear acceleration A in three dimensions. x , A y , A z The 6-DOF motion detection device can be combined with a multi-area pressure sensing device (e.g., multiple pressure sensors in an insole) to provide dynamic tracking of foot movements in space and time that can be mapped during walking / exercise.

[0029] FIG. 3 illustrates a functional block diagram of the sensor assembly 102, including the inertial sensing module 216 capable of transmitting and receiving data via a wireless transceiver (TX / RX) 232. While FIG. 3 illustrates the wireless transceiver (TX / RX) 232 being integrated into the inertial sensing module 216, those skilled in the art will appreciate that the wireless transceiver (TX / RX) 232 may also be used as a separate assembly for data transmission and reception. In the example of FIG. 3, the inertial sensing module 216 may include the wireless transceiver (TX / RX) 232 for transmitting data to and / or receiving data from one or more remote systems. In one embodiment, the wireless transceiver 232 may be a low-power, medium- to long-range network connection device, such as a Bluetooth chip (Bluetooth is a registered trademark), WiFi, RF, Zigbee, Narrowband IoT (NB-IoT), or similar wireless transceiver. The inertial sensing module 216 can be electrically connected to multiple pressure sensors (pressure sensing device 238) provided in the insole via connection terminals. The inertial sensing module 216 further includes a processor (e.g., one or more microprocessors 234), a memory 235, a composite sensor 216-1 (including an accelerometer and a gyroscope (G-sensor)), a GPS sensor 216-2, and a power supply 237. The power supply 237 can supply electricity to the pressure sensing device 238 and / or other devices of the sensor assembly 102 (e.g., the microprocessor 234, the memory 235, the composite sensor 216-1, the GPS sensor 216-2, etc.). In a preferred embodiment, the power supply 237 includes a rechargeable solid-state battery, an inductor coil (used to be connected to an external wireless charging system to wirelessly charge the battery), and a USB charging port.It should be understood that the sensor assembly 102 is capable of collecting and storing data relating to the user's physical movements using computer programs / algorithms (e.g., pressure distribution data of the user's feet or pressure data due to interaction with the ground (e.g., impact force, etc.), the user's movement trajectory (forward, backward, jump), speed / distance, acceleration, related data of angular direction and changes in angular direction, etc.), and is capable of storing and / or executing these programs / algorithms.

[0030] The wireless transceiver 232 completes the connection of one or more sensors and can provide additional composite sensors 216-1, GPS sensors 216-2, etc. to detect and provide data or information on various different parameters. This data or information can include physiological data about the user, pressure distribution data on the user's feet or pressure data due to interaction with the ground, user's movement trajectory (forward, backward, jump), speed / distance, GPS data, acceleration output / data, related data of angular orientation and change in angular orientation (obtained by gyroscope sensors). This data can be stored in memory or transmitted via the wireless transceiver 232 to a remote terminal computing electronic device or server.

[0031] The inertial sensing module 216 may further be configured to communicate with an external computing electronic device 104. The external computing electronic device 104 may include a computing device such as a smartphone, a laptop, a tablet PC, or a personal computer, for example.

[0032] From the perspective of the system, as shown in FIG. 4, one user 108 uses sensor assemblies 102 on both feet (foot parts), one on each foot (for example, sensor modules (102a, 102b) provided in the left and right insoles, respectively).

[0033] FIG. 4 shows a schematic system block diagram of communication between the sensor modules (102a, 102b) and the external computing electronic device 104. The sensor modules (102a, 102b) mounted on the feet (e.g., left and right insoles) of a user 108 each include an inertial sensing module (216a, 216b) embedded in the arch of the insole. The inertial sensing module (216a, 216b) is electrically connected to a pressure sensing device (238a, 238b) for receiving and analyzing the user's foot pressure distribution data and transmitting the data to a remote terminal computing device or server via a wireless transceiver (232a, 232b) located within the sensing module. Each of the inertial sensing modules (216a, 216b) includes a processor (e.g., one or more microprocessors), memory, additional sensors, and a power supply (see FIG. 2).

[0034] The external computing electronic device 104 may be any electronic device capable of transmitting, processing, and / or storing data. In one embodiment, the external computing electronic device 104 is a portable computing device. The portable computing device may be a social networking device, a gaming device, a mobile phone, a smartphone, a personal digital assistant, a digital audio / video player, a laptop, a tablet PC, a video game controller, and / or any other portable device that includes a computing core.

[0035] The external computing electronic device 104 includes a computing core 342, a user interface 343, an internet interface 344, a wireless communications transceiver 345, and storage 346. The user interface 343 includes one or more input devices (e.g., keyboard, touch screen, audio input device, etc.), one or more audio output devices (e.g., speakers, earphone jack, etc.), and / or one or more visual output devices (e.g., video graphic display, touch screen, etc.). The internet interface 344 includes one or more network connection devices (e.g., wireless local area network (WLAN) device, wired local area network (WWAN) device, etc.). The storage 346 includes a flash memory device, one or more hard disk drives, one or more solid-state (SS) storage devices, and / or cloud storage.

[0036] Computing core 342 includes processor 342a and other computing core components 342b, which may include a video and graphics processing unit, a memory controller, main storage (e.g., RAM), one or more input / output (I / O) device interface modules, input / output (I / O) interfaces, input / output (I / O) controllers, peripheral interfaces, one or more USB interface modules, one or more network interface modules, one or more storage device interface modules, and / or one or more peripheral interface modules.

[0037] The wireless communications transceiver 345 of the computing electronic device 104 and the wireless transceivers (232a, 232b) of the sensor modules (102a, 102b) have similar transceiver types (e.g., Bluetooth, WLAN, Wi-Fi, etc.). The wireless transceivers (232a, 232b) communicate directly with the wireless communications transceiver 345 to share collected data with their respective sensor modules (102a, 102b) and / or receive instructions from the external computing device 104. Additionally or alternatively, the wireless transceivers (232a, 232b) exchange collected data with each other and transmit aggregated data to the wireless communications transceiver 345 of the external computing electronic device 104.

[0038] According to an embodiment of the present invention, referring to FIGS. 1-4, when a user 108 steps on a pressure sensor, the pressure sensing devices (238a, 238b) are activated. Also, when the user takes a step, jumps, or runs, the inertial sensing modules (216a, 216b) are activated. Both the pressure sensing devices (238a, 238b) and the inertial sensing modules (216a, 216b) transmit electrical signals to the external computing electronic device 104. The microprocessor 234 of the sensor assembly 102 compiles data about the user's foot during exercise, such as a pressure wave curve (a curve of pressure change over time) and an acceleration curve (A) of the user's foot. x , A y , A z curve of time-dependent change of ω x , ω y , ω z The pressure wave and acceleration curves can be used to detect and analyze the peak values ​​of the pressure, acceleration, or angular velocity curves, and then convert these continuous pressure waves and acceleration curves into discrete pulse signals, which are then numbered in time series to form corresponding time series pulse signals of pressure, acceleration, or angular velocity.

[0039] According to an embodiment of the present invention, the gyroscopes of the inertial sensing modules (216a, 216b) measure acceleration curves A along the X, Y, and Z axes. x , A y and A z By integrating these in order, the velocity V of the X, Y and Z axes can be calculated. x , V y and V z The gyroscopes of the inertial sensing modules (216a, 216b) can obtain the angular velocity ω of rotation around the X, Y, and Z axes. x , ω y and ω z By sequentially integrating these, the pitch angle (θ), roll angle (γ), and yaw angle (ψ) can be obtained.

[0040] For example, in typical physical activities such as walking, dancing, or playing ball games, people alternate between moving their left and right feet. To generate musical notes from these alternating footsteps, the pulses of each signal transmitted from the wireless transceivers (232a, 232b) of the sensor modules (102a, 102b) are all numbered in order.

[0041] Regarding how information about the user's gait is analyzed from the pressure wave curve (change in pressure over time) detected by the pressure sensing devices (238a, 238b), for example, the length of the user's 108 stride time can be measured from the time difference between successive heel strikes of the same foot of the user 108, the user's swing time (i.e., the length of time the foot swings in the air) can be measured from the time difference between toe off and heel strike of the same foot of the user, and the user's stand-up time (i.e., the length of time the foot is in contact with the floor) can be measured from the time difference between heel strike and toe off. This data may be combined with data received by the inertial sensing modules (216a, 216b) measurement units and GPS sensor 236 to enable measurement of step length or stride length and other performance characteristics of cadence / gait.

[0042] According to an embodiment of the present invention, the heel strike pressure signal corresponds to the foot pressure distribution signal detected by the pressure sensor provided in the heel region 220 shown in Figure 2. The toe off pressure signal corresponds to the foot pressure distribution signal detected by the pressure sensor provided in the toe region 215 shown in Figure 2.

[0043] As mentioned in the above paragraph, by analyzing the pressure wave curve (change in pressure over time) of the user's 108 feet, information such as the number of strikes, stride length, step length, pitch, and standing time can be obtained. Furthermore, by combining this information with data received by the accelerometers and GPS sensors of the inertial sensing modules (216a, 216b), further parameters related to the user's gait, such as forward movement, backward movement, jumping, and movement trajectory, can be obtained. In addition, the rotational movement of the user's feet can be further obtained from data received by the gyroscopes of the inertial sensing modules (216a, 216b).

[0044] Please refer to Fig. 5, which shows a functional block diagram corresponding to the system configuration of Fig. 1. The external computing electronic device 104 receives time-series pulse signals of pressure and acceleration, which are data related to the movement of the user 108, and generates various gait parameters by processing the data using a gait analysis module 340. For example, the processor 342a of the external computing electronic device 104 is combined with an arithmetic circuit or algorithm of the gait analysis module 340 to process data related to the movement of the user 108, such as time-series pulse signals of pressure, acceleration, and angular velocity, from the sensor modules (102a, 102b). The external computing electronic device 104 analyzes the collected user movement data using a gait analysis algorithm provided in a gait analysis module 340 to analyze time-series pulse signals of the pressure, acceleration, and angular velocity of the user's 108's left and right feet. By combining this with the collected GPS data, the external computing electronic device 104 can acquire and output gait parameter information, such as the number of times the user 108 kicks the ground, stride length, step length, pitch, standing time, jump, forward movement, backward movement, foot rotation motion, and movement trajectory. These output gait parameters are then passed through a rhythm parameter extraction module 350, which extracts related movement rhythm parameters, such as rhythm, speed, jump height, force magnitude, stride length and foot rotation width, and frequency. These movement rhythm parameters can be transmitted via the Internet to an AI music generation module 360 ​​in the cloud server 106. These related movement rhythm parameters, such as rhythm, speed, jump height, force magnitude, stride length and foot rotation width, and frequency, are then converted into corresponding beats, pitch, weight, etc., and a melody is added using an AI algorithm to generate and output music. This allows AI to generate music related to exercise.

[0045] According to an embodiment of the present invention, the music generated by the AI ​​music generation module can be interacted with by a user through a music-footing interaction module 370 provided in the external computing electronic device 104. For example, the music-footing interaction module 370 may be an application (APP) that can be created as a group online game with elaborate rehabilitation functions, with different levels of rehabilitation depending on game difficulty and scoring mechanisms. The group online game may include a mobile phone dance machine application (APP). The application may have, for example, moving graphics generated in sync with the rhythm of the music generated by the AI ​​music generation module 360 ​​and be used to interact with a user, for example, by training the user to perform rehabilitation in sync with the rhythm of their footsteps (generated by the mobile phone dance machine).

[0046] According to an embodiment of the present invention, the AI ​​music generation module 360 ​​may include similar AI music generators such as Amper Music, AVIA, Ecrett Music, Soundraw, Boomy, MuseNet, or Amadeous2 / 3xo4n42k7, which can generate music through machine learning.

[0047] According to an alternative embodiment of the present invention, a system configuration corresponding to Figure 5 may include wireless signals generated by the physical movements of multiple individuals (multiple users), including corresponding pressure, acceleration, and GPS data detected by sensor assemblies provided on corresponding feet of the multiple users, and such multiple individuals may be, for example, members of a dance group or participants in a sports competition.

[0048] For example, a multi-user system may have a group of people, each member equipped with a corresponding sensor assembly, and each sensor assembly may have a wireless transceiver unit, each corresponding sensor and wireless transceiver unit capable of communicating with the wireless communications transceiver 345 of the computing electronic device 104.

[0049] The above description is a preferred embodiment of the present invention. Those skilled in the art should understand that the above is for the purpose of illustrating the present invention and does not limit the scope of the rights claimed in the present invention. The scope of protection of the rights is determined based on the scope of the claims set forth below and their equivalent fields. Any modifications or additions made by those skilled in the art without departing from the spirit or scope of the patent are considered equivalent variations or designs made in the spirit disclosed in the present invention and should be included in the scope of the following claims. [Explanation of symbols]

[0050] 100 System Configuration 102 Sensor Assembly 102a, 102b Sensor modules 104 Computing Electronic Devices 106 Cloud Server 108 users 210 Insole 214 Arch Pad 215 Area near the toe of the shoe 216 Inertial Sensing Module 216-1 Composite Sensor 216-2 GPS sensor 216a, 216b Inertial sensing module 219 Lateral arch area 220 Heel area 222 Conductor 224 Wiring 232 Wireless Transceiver 232a, 232b Wireless transmitter / receiver 234 microprocessor 235 memory 237 Power supply equipment 238 Pressure Sensing Device 238a, 238b Pressure sensing device 340 Gait Analysis Module 342 computing cores 342a processor 342b Other Computing Core Components 343 User Interface 344 Internet Interface 345 Wireless communication transceiver 346 Storage device 350 Rhythm Parameter Extraction Module 360 AI Music Generation Module 370 Music and Footstep Interaction Module

Claims

1. A gait-based artificial intelligence (AI) music generation device, comprising: a sensor assembly provided on the user's foot, the sensor assembly including a pressure sensing device and an inertial sensing module, configured to collect data regarding the user's movement; a computing electronic device communicatively connected to the sensor assembly, the computing electronic device including a gait analysis module and a rhythm parameter extraction module, the gait analysis module being used to analyze data related to the movement to obtain gait parameters, and the rhythm parameter extraction module being used to extract rhythm parameters related to the movement; an AI music generation module provided on a cloud server or the computing electronic device, communicatively connected to the computing electronic device and configured to receive rhythmic parameters related to the exercise, convert the rhythmic parameters related to the exercise into corresponding beats, pitch, and weight, and add a melody to generate music and output it to the computing electronic device.

2. 2. The gait-based artificial intelligence (AI) music generation device of claim 1, further comprising: a music-gait interaction module provided in the computing electronic device, which generates graphics that move in sync with the rhythm of the music generated by the AI ​​music generation module and is used to interact with the user.

3. 2. The gait-based artificial intelligence (AI) music generation device of claim 1, wherein the pressure sensing device includes a plurality of pressure sensors used to detect pressure distribution in the toe region and the heel region, respectively, and the inertial sensing module includes one or a combination of an accelerometer and a gyroscope, and is used to detect the movement of the user's feet.

4. 2. The gait-based artificial intelligence (AI) music generation device of claim 1, wherein the gait parameters include one or any combination of the user's ground kick count, stride length, step length, pitch, standing time, forward movement, backward movement, jump, and movement trajectory.

5. 2. The gait-based artificial intelligence (AI) music generation device of claim 1, wherein the rhythmic parameters of the movement include one or any combination of rhythm, speed, jump height, force magnitude, stride length and gait rotation width, and frequency.

Images