AI control system based on intelligent musical instrument

Abstract

The invention discloses an AI control system based on an intelligent musical instrument, and relates to the technical field of electronic musical instrument control, and the AI control system comprises a voice collection module, an AI interaction processing module and an intelligent musical instrument main control chip, and supports an online control mode and an offline control mode. A user voice instruction is collected and recognized through an INMP441-ICS434 voice module, a control instruction is generated through deep analysis of a bean bag AI or a Deepseek model (online connection with cloud service and offline calling of a lightweight model), and an intelligent musical instrument main control chip controls a musical instrument to execute functions of tone switching, rhythm adjustment, playing teaching follow-up and the like. Complex natural language interaction and real-time dialogue are supported, fuzzy instructions can be understood, active inquiry and clarification can be achieved, resources can be downloaded online, pre-stored resources can be called offline, the operation convenience and the teaching intelligence level of the intelligent musical instrument are remarkably improved, the use scene is expanded, and the method is suitable for various electronic organs, portable electric pianos and other intelligent musical instruments.

Description

Technical Field

[0001] This invention relates to the field of electronic musical instrument control technology, and more specifically to an AI-based control system for intelligent musical instruments. Background Technology

[0002] Traditional musical instruments are operated primarily by manually pressing buttons or knobs to select functions such as tone and rhythm. This method of operation presents many inconveniences for users, especially beginners. For example, finding the appropriate setting among various tone and rhythm options is time-consuming and cumbersome, easily distracting the player and affecting the performance experience.

[0003] In terms of instructional practice and follow-along functionality, traditional methods typically require specific apps with complex operating procedures. Learners not only need to search for suitable lessons within the app but also perform a series of operations such as connecting devices and setting parameters. This is undoubtedly a significant hurdle for beginners unfamiliar with electronic devices, greatly reducing their motivation and efficiency in learning.

[0004] With the development of technology, some methods for controlling musical instruments using artificial intelligence through mini-programs have emerged in the market. While these methods improve the intelligence of instrument control to some extent, they also have significant drawbacks. Firstly, mini-programs rely on mobile devices such as smartphones to operate. If the phone battery runs out, the network signal is poor, or the device malfunctions, the AI ​​control function of the instrument will be unavailable, severely impacting the continuity and stability of user experience. Secondly, mini-program operations still require clicking on the phone screen, making it difficult for users to simultaneously manage phone operation and instrument playing during performance, also causing inconvenience. Furthermore, most of these control methods only support online mode and cannot function properly in environments without a network connection, limiting their application scenarios. Summary of the Invention

[0005] The purpose of this invention is to provide an AI control system based on intelligent musical instruments to solve the problems of inconvenient operation of intelligent musical instruments, reliance on external devices and networks, and complex use of teaching functions in the prior art, thereby improving the ease of operation and intelligent teaching of intelligent musical instruments and expanding the application scenarios.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] An AI control system based on an intelligent musical instrument comprises: a voice module, an AI interaction processing module, and an intelligent musical instrument main control chip. The voice module, using an INMP441-ICS434 voice module, is used to collect user voice signal commands for preprocessing, feature extraction, and recognition, supporting both online and offline recognition. The AI ​​interaction processing module connects the INMP441-ICS434 voice module and the intelligent musical instrument main control chip, transmitting command text and control commands, which are then parsed by the intelligent musical instrument main control chip. The AI ​​interaction processing module is equipped with a Doubao AI or online interface and a lightweight offline model for deep analysis of command text, generating control commands, enabling real-time dialogue interaction, and supporting both online and offline working modes. In online mode, it connects to the Doubao AI or... The DeepSeek service, in offline mode, uses an internal lightweight model to parse commands. It can understand complex and ambiguous natural language commands in both modes, generating corresponding control commands. When a command is unclear, it can proactively ask the user for accurate information. For example, when a user says "play a cheerful tune," the module can quickly understand the emotional requirement of "cheerful" and, combined with its understanding of musical styles, select suitable accompaniment tracks from its music library. In offline mode, its lightweight model, based on learning from a large number of common musical commands, can also effectively parse similar commands, providing users with accurate control commands. The intelligent musical instrument's main control chip, after processing the parsed control commands, executes timbre control, rhythm control, and playing tutorials, supporting online resource acquisition and offline retrieval of pre-stored resources.

[0008] Specifically, the timbre control involves the intelligent musical instrument's main control chip using control commands generated by the AI ​​interaction processing module to retrieve timbre data from the instrument's internal memory. It then simulates different instrument timbres by changing audio signal parameters and outputs the sound through an audio amplification circuit. For example, when a user issues the command "switch to piano timbre," the main control chip responds quickly, retrieves piano timbre data from memory, and accurately simulates the unique timbre of a piano by adjusting parameters such as the frequency, amplitude, and harmonics of the audio signal. The sound is then output through the audio amplification circuit.

[0009] The rhythm control specifically involves the intelligent musical instrument's main control chip reading rhythm data from the instrument's rhythm data storage area through control commands generated by the AI ​​interaction processing module. This allows the chip to add embellishments and variations to the instrument's rhythm, adjust the tempo and stop the rhythm via commands, and then mix and output the rhythm accompaniment through an audio synthesis circuit. For example, if a user requests "speed up the rhythm and add some embellishments," the main control chip will find the corresponding rhythm data from the rhythm data storage area, increase the tempo, and perform variations according to preset embellishment rules. Finally, the accompaniment is mixed with other audio elements through the audio synthesis circuit to output a rhythm accompaniment that meets the user's requirements.

[0010] The accompanying practice function works as follows: The intelligent musical instrument's main control chip initiates the practice program via control commands generated by the AI ​​interaction processing module, plays a demonstration piece, and records its note sequence and rhythm information. While the user plays, the chip compares the user's playing with the demonstration piece and provides feedback control based on the comparison results. Users can also switch demonstration pieces via commands. For example, when a user activates the practice function and selects a demonstration piece, the main control chip starts playing the demonstration piece and simultaneously records its note sequence and rhythm information. During the user's playing, the chip collects the user's notes and rhythm in real time and compares them frame-by-frame with the demonstration piece. If an error is detected, feedback is provided promptly through indicator light flashing, voice prompts, etc., guiding the user to correct the mistake. Simultaneously, the user can issue commands such as "switch to the next demonstration piece" at any time, and the main control chip will respond quickly and execute the corresponding operation.

[0011] Furthermore, the output of the voice acquisition module is connected to the input of the AI ​​interaction processing module via an I2S audio interface to realize the transmission of voice command text.

[0012] Furthermore, the output of the AI ​​interaction processing module is connected to the input of the smart musical instrument main control chip via a UART serial communication interface, which is used to transmit the parsed and generated control commands to the main control chip.

[0013] Furthermore, the intelligent musical instrument's main control chip is connected to the instrument's internal memory, audio amplification circuit, audio synthesis circuit, indicator light control circuit, speech synthesis circuit, and sensors via an SPI bus or GPIO interface. The connection to the memory is used to retrieve timbre data, rhythm data, and demonstration song data; the connection to the audio amplification circuit and audio synthesis circuit is used to output processed sound signals; the connection to the indicator light control circuit and speech synthesis circuit is used to provide feedback prompts in the playing and teaching follow-along function; and the connection to the sensors is used to collect user playing information.

[0014] Furthermore, the AI ​​interaction processing module also establishes a network connection with the cloud service through an Ethernet interface or a Wi-Fi module to achieve data interaction in online mode. Beneficial effects of the invention

[0015] The ease of use has been greatly improved. Users can easily control the smart instrument through voice commands, eliminating the need for tedious manual operations during performance, freeing up their hands and making the playing process smoother and more natural. It is especially suitable for use in performance or improvisation scenarios. The playing instruction and follow-along function is simple to operate and provides real-time feedback based on the user's playing, helping users quickly improve their playing level, lowering the learning threshold and increasing learning efficiency. It is particularly suitable for beginners of musical instruments. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the architecture of the AI ​​control system for intelligent musical instruments based on the present invention.

[0017] As shown in the figure, the voice module is connected to the AI ​​interaction processing module via a data transmission line, and the AI ​​interaction processing module is then connected to the main control chip of the smart musical instrument via a control bus. The voice module is responsible for collecting user voice commands, the AI ​​interaction processing module parses and processes the commands, and the main control chip of the smart musical instrument controls the instrument to perform corresponding functions based on the processed commands. Detailed Implementation

[0018] Voice module workflow:

[0019] When a user issues a voice command, the INMP441-ICS434 voice module begins operation. First, the microphone array collects ambient sound. The raw voice signal is initially amplified by a preamplifier to increase signal strength. Next, the signal enters a filter bank. A high-pass filter removes low-frequency noise, such as ambient hum, while a low-pass filter removes high-frequency interference, such as high-frequency noise from electronic devices. The filtered signal then enters an A / D converter, converting the analog signal to a digital signal for subsequent digital signal processing. Feature extraction is performed on the digital signal in the voice recognition chip, using algorithms such as Mel-frequency cepstral coefficients (MFCC) to extract characteristic parameters of the voice signal. The extracted feature parameters are then matched against a built-in voice recognition model (online or offline). For online recognition, the chip sends the feature parameters to a cloud server via the network, utilizing the cloud's powerful computing resources and large-scale voice recognition models for recognition. The recognition result is returned as a command text. For offline recognition, the chip directly uses a small built-in voice recognition model to recognize the feature parameters, outputs the command text, and transmits it to the AI ​​interaction processing module.

[0020] AI Interaction Processing Module Workflow:

[0021] After receiving the command text from the voice module, the AI ​​interaction processing module first determines the current operating mode. In online mode, the module connects to the cloud-based Doubao AI or Deepseek service via the network; the command text is sent to the cloud model, which utilizes its rich knowledge base and powerful language understanding capabilities to perform deep analysis of the command. For example, for a command like "play an instrumental version of a Jay Chou-style pop song," the cloud model understands the musical characteristics of "Jay Chou style," such as rhythmic features and harmonic style, and selects suitable instrumental tracks from a massive music resource library, generating corresponding control commands, such as track numbers and playback control instructions, before sending these commands back to the AI ​​interaction processing module. In offline mode, the AI ​​interaction processing module activates its internal lightweight model. This model parses the command text based on learning from a large number of common music commands. For example, for a command like "switch to rock rhythm," the lightweight model understands the keyword "rock rhythm" in the command, finds the corresponding rhythm data number from the locally stored rhythm type library, generates a control command, and instructs the smart instrument's main control chip to retrieve the corresponding rock rhythm data. During the instruction parsing process, if the model encounters an ambiguous instruction, such as "Give me a special timbre," the module will use speech synthesis technology to ask the user, such as "What style of special timbre do you want, classical, pop, or folk style?" After obtaining accurate instruction information, it will parse and generate control instructions, and then send the control instructions to the smart musical instrument's main control chip.

[0022] Working process of intelligent musical instrument main control chip:

[0023] After receiving control commands from the AI ​​interaction processing module, the main control chip of the instrument performs corresponding operations according to the command type.

[0024] In terms of timbre control, if the control command is to switch timbres, the main control chip reads the corresponding timbre data from the instrument's internal memory based on the timbre number carried in the command. For example, if the timbre data is stored in the form of a waveform table, the main control chip reads the waveform table data and then processes the audio signal through a digital signal processor (DSP). Based on the characteristics of different timbres, parameters such as the frequency, amplitude, and phase of the audio signal are adjusted. For instance, when simulating a violin timbre, the harmonic distribution of the signal is adjusted to make it closer to the timbre characteristics of a real violin. The processed audio signal is converted into an analog signal by a digital-to-analog converter (DAC), then amplified by an audio amplifier circuit, and finally drives the speaker to output sound.

[0025] In terms of rhythm control, if the control command is to adjust the rhythm, the main control chip reads the rhythm data from the instrument's rhythm data storage area. This area stores data according to different rhythm types, speeds, and other parameters. The main control chip finds the corresponding rhythm data based on the rhythm type and speed requirements in the command. For example, for a command to "increase the rhythm speed by 20%", the main control chip reads the current rhythm data and then increases the playback speed by adjusting the frequency division of the rhythm clock signal. If the command includes embellishment or variation requirements, the main control chip modifies the rhythm data according to preset embellishment rules. For instance, in a common 4 / 4 time signature, additional percussion effects can be added to the weak beats of specific measures to achieve rhythm embellishment or variation. The processed rhythm data is then mixed with other audio elements (such as melody and harmony) through an audio synthesis circuit to output a complete rhythmic accompaniment.

[0026] Regarding the accompaniment function for playing instruction, if the control command is to start the accompaniment instruction program, the main control chip first reads the user-selected demonstration piece data from the storage device. The demonstration piece data includes the audio file and the corresponding note sequence and rhythm information. After decoding the audio file, the main control chip plays the demonstration piece through the audio output circuit, while storing the note sequence and rhythm information in the cache. During the user's playing, the main control chip collects the note and rhythm information played by the user in real time through sensors. For example, for keyboard instruments, it obtains note and rhythm information by detecting the pressing and releasing time of the keys. The user's playing information is then compared with the notes of the demonstration piece. The sequence and rhythm information are compared frame by frame. If the notes played by the user are inconsistent with the demonstration piece, the main control chip illuminates the corresponding indicator light through the indicator light control circuit to alert the user to the error position. At the same time, the voice synthesis circuit issues a voice prompt, such as "The third note of this measure is incorrect, please play it again." If the rhythm played by the user is inconsistent with the rhythm of the demonstration piece, the main control chip issues a rhythm correction prompt through the audio output circuit based on the rhythm deviation to help the user adjust the rhythm. When the user issues a command to switch demonstration pieces, the main control chip stops the playback and follow-along comparison of the current demonstration piece and reads and plays the new demonstration piece according to the above process.

[0027] The above description is merely a preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. It should be noted that modifications or equivalent substitutions to the technical solutions of the present invention by those skilled in the art do not depart from the protection scope of the present invention.

Claims

1. An AI control system based on intelligent musical instruments, characterized in that, include: It consists of a voice module, an AI interaction processing module, and a smart musical instrument main control chip; The voice module uses the INMP441-ICS434 voice module to collect user voice signal commands for preprocessing, feature extraction and recognition, and supports online and offline recognition. The AI ​​interaction processing module connects the INMP441-ICS434 voice module and the main control chip of the smart musical instrument. It is used to transmit instruction text and control commands, and the main control chip parses the control commands. The AI ​​interaction processing module is equipped with Doubao AI or online interface and lightweight offline model, which is used to perform deep analysis of instruction text and generate control commands. It can realize real-time dialogue interaction and supports both online and offline working modes. The intelligent musical instrument main control chip executes tone control, rhythm control, and playing and teaching functions through the parsed control instructions, and supports online resource acquisition and offline retrieval of pre-stored resources.

2. The AI ​​control system based on intelligent musical instruments according to claim 1, characterized in that, The AI ​​interaction processing module connects to the cloud-based Doubao AI or Deepseek service in online mode, and performs command parsing through an internal lightweight model in offline mode. It can understand complex and ambiguous natural language commands in both modes and generate corresponding control commands. When the command is unclear, it can actively ask the user for accurate information.

3. The AI ​​control system based on intelligent musical instruments according to claim 1, characterized in that, The timbre control specifically involves the intelligent musical instrument main control chip calling the timbre data in the instrument's internal memory through control commands generated by the AI ​​interaction processing module, simulating different instrument timbres by changing audio signal parameters, and outputting sound through the audio amplification circuit.

4. The AI ​​control system based on intelligent musical instruments according to claim 1, characterized in that, The rhythm control specifically involves the intelligent musical instrument main control chip reading rhythm data from the instrument's rhythm data storage area through control commands generated by the AI ​​interaction processing module, controlling the instrument's rhythm to add embellishments and variations, adjusting the rhythm speed and stopping through commands, and then mixing and outputting the rhythm accompaniment through the audio synthesis circuit.

5. The AI ​​control system based on intelligent musical instruments according to claim 1, characterized in that, The specific function of the playing instruction and follow-along function is as follows: the main control chip of the intelligent musical instrument starts the follow-along teaching program through the control command generated by the AI ​​interaction processing module, plays the demonstration piece and records its note sequence and rhythm information, and provides feedback control by comparing the user's playing information with the demonstration piece when the user plays, and the demonstration piece can be switched by command.

6. The AI ​​control system based on intelligent musical instruments according to claim 1, characterized in that, The output of the voice module is connected to the input of the AI ​​interaction processing module via an I2S audio interface to transmit voice command text.

7. The AI ​​control system based on intelligent musical instruments according to claim 1, characterized in that, The output of the AI ​​interaction processing module is connected to the input of the smart musical instrument main control chip via a UART serial communication interface, which is used to transmit the parsed and generated control commands to the main control chip.

8. The AI ​​control system based on intelligent musical instruments according to claim 1, characterized in that, The intelligent musical instrument's main control chip is connected to the instrument's internal memory, audio amplification circuit, audio synthesis circuit, indicator light control circuit, speech synthesis circuit, and sensors via an SPI bus or GPIO interface. The connection to the memory is used to retrieve timbre data, rhythm data, and demonstration song data; the connection to the audio amplification circuit and audio synthesis circuit is used to output processed sound signals; the connection to the indicator light control circuit and speech synthesis circuit is used to provide feedback prompts in the playing and teaching follow-along function; and the connection to the sensors is used to collect user playing information.

9. The AI ​​control system based on intelligent musical instruments according to claim 1, characterized in that, The AI ​​interaction processing module also establishes a network connection with the cloud service through an Ethernet interface or a Wi-Fi module to achieve data interaction in online mode.

Images