Universal musical instrument audio data processing system for supporting MIDI protocol

Abstract

The invention discloses a universal musical instrument audio data processing system for supporting an MIDI protocol, and relates to the field of electronic musical instruments. Comprising an annular buffer area module, a serial port receiving and transmitting module, a USB receiving and transmitting module, an MIDI stream decoding module, a common MIDI command processing module and an SYSEX MIDI command processing module, the SYSEX MIDI command processing module is provided with an APP / upper computer communication protocol, a Bluetooth module communication protocol, a sound production engine module, a user data access module, a character terminal module, an MIDI command packaging module, an MIDI command sending module and a service processing module. A friendly man-machine interaction interface is designed for the universal musical instrument audio data processing system, so that an application engineer can concentrate on development of product functions, energy investment on system framework building is reduced, the product development period is greatly shortened, design efficiency is improved, test cost investment is greatly reduced, and the system is suitable for popularization and application. And risks of product repair, customer complaint and the like are reduced.

Description

Technical Field

[0001] The present invention relates to the field of electronic musical instruments, and in particular to a general musical instrument audio data processing system for supporting the MIDI protocol. Background Art

[0002] With the continuous development of the functions of electronic musical instruments and the popularization of digital audio workstation (DAW) software, more DAW software relies on the MIDI protocol to connect with electronic musical instruments to control the sound production, parameter settings, service interactions, etc. of electronic musical instruments. Since there is no general framework, the system is built from scratch according to the product function requirements during development. Moreover, since most development engineers (application engineers) are from computer majors and do not have enough knowledge of music theory and MIDI protocol, there are many improper conditional processes during MIDI encoding and decoding, resulting in poor compatibility. The system framework design is unreasonable or the scalability is poor, making it difficult to reuse on different types of musical instruments or different platforms.

[0003] In the current musical instrument product chips and development solutions, the French manufacturer Dream (DREAM-Audio applications) provides a complete set of chip solutions and development SDK packages. Especially for its Sam5000 series, the development SDK package already includes MIDI encoding and decoding, MIDI file playback, sound production engine, message processing framework, hardware drivers, etc., which are all made and packaged into library files. Manufacturers can quickly complete product development and go to market after getting the development SDK package, so it is adopted by many musical instrument manufacturers. However, due to the deep binding of the development SDK package with the chip hardware, it cannot be transplanted to other chip platforms, and most of the development SDK packages are released in the form of libraries. It is difficult for manufacturers to make product differentiation in terms of functions, resulting in similar and single functions of products on the market. With the continuous development of MIDI applications, many new effect algorithms, hybrid effect links and other advanced applications have emerged. However, due to reasons such as the low computing power, few peripheral resources and high algorithm development difficulty of Dream chips, it is difficult to meet the needs of new product applications. At the same time, due to the lack of a good human-computer interaction method, software debugging is not convenient, and it is difficult to analyze the reasons for problems after the product reaches the end customers. The present invention provides a general software framework and the implementation of functional codes, which can be easily transplanted to different chip platforms and can be freely trimmed and modified according to the product function and performance requirements.

[0004] Therefore, the present invention proposes a general musical instrument audio data processing system for supporting the MIDI protocol. Summary of the Invention

[0005] The purpose of the present invention is to solve the defects existing in the prior art, and to propose a general musical instrument audio data processing system for supporting the MIDI protocol.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A general musical instrument audio data processing system for supporting the MIDI protocol, comprising:

[0008] A circular buffer module, which is used to cache data from interrupts or another thread and serves as the main data interaction carrier between threads;

[0009] A serial port transceiver module, which receives data from the serial port bus through DMA and writes it into the receive circular buffer, activates the thread of the serial port transceiver module to call the MIDI stream decoding module through a semaphore; reads the data in the transmit circular buffer and sends it to the serial port bus through DMA;

[0010] A USB transceiver module, which receives or sends MIDI stream data through the USB bus;

[0011] A MIDI stream decoding module, which decodes the MIDI data stream according to the standard MIDI protocol and identifies the instruction type and its parameters according to the system code type;

[0012] An ordinary MIDI command processing module, which processes ordinary MIDI commands and is used to control the sound engine module to emit specified musical instrument sounds;

[0013] A SYSEX MIDI command processing module, which processes private communication protocol data and is used to control the working parameters of the device and handle the service interaction after the mobile phone APP is connected;

[0014] The SYSEX MIDI command processing module supports the APP / host computer communication protocol and the Bluetooth module communication protocol;

[0015] A sound engine module, which adopts the PING-PONG double buffer mode and outputs the audio data in the PING buffer to the I2S peripheral through DMA;

[0016] A user data access module, which is used for accessing user data;

[0017] A character terminal module, which is used to receive user input instructions and execute corresponding command processing after completing instruction interpretation;

[0018] A MIDI command packetizing module, which supports packetizing of ordinary MIDI commands, SYSEX MIDI commands, and USB MIDI 4-byte protocol data groups, and appends the packetized data frames to the send waiting queue and then activates the thread of the MIDI command sending module;

[0019] A MIDI command sending module, which is responsible for managing the data frame sending queue and command response processing;

[0020] The service processing module, after the application engineer completes corresponding input / output detection, interaction logic, service functions, user settings, etc. according to the product functions, generates corresponding ordinary MIDI codes and sends them to the sound generation engine module to output sounds, calls the MIDI command packaging module to send MIDI protocol commands to the serial port, USB or Bluetooth module, reads or saves user data to the storage medium through the user data access module, and receives debugging instructions or outputs debugging information through the character terminal module.

[0021] Preferably, the circular buffer module is built-in with a read semaphore mechanism and a write lock mechanism.

[0022] Preferably, the logic of the read semaphore mechanism is as follows: When thread A reads the circular buffer, if the buffer data is empty or the amount of data is insufficient for the requested amount, thread A will be suspended and the CPU resources will be released; until the buffer reaches the requested amount of data or the timeout, thread A will be reactivated.

[0023] Preferably, the logic of the write lock mechanism is as follows: When thread B writes to the circular buffer, it will lock the buffer. If thread C also operates on the same buffer at this time, it will be suspended until thread B releases the lock, and then thread C will be activated.

[0024] Preferably, the receiving process of the USB transceiver module is as follows: The USB triggers an interrupt, and the interrupt program reads 4 bytes of data from the receiving endpoint, determines the subsequent valid data length by judging the CIN code of Bit0...3 of Byte0, writes the valid data into the receiving circular buffer and activates the USB transceiver module thread;

[0025] The sending process of the USB transceiver module is as follows: When the host initiates a data query instruction interrupt, the interrupt program reads up to 64 bytes of data from the sending circular buffer and sends it to the USB bus.

[0026] Preferably, in the MIDI stream decoding module:

[0027] If it is an ordinary MIDI command, it triggers the call to the ordinary MIDI command processing module;

[0028] If it is a SYSEX extension instruction, it calls the SYSEX MIDI command processing module.

[0029] Preferably, the working logic of the sound generation engine module is as follows: When DMA completes a data transfer, it triggers an interrupt, outputs the audio data in the PONG buffer to the I2S peripheral through DMA, and at the same time sends an activation event to the sound generation engine module thread to trigger the execution of a sound generation and effect processing, and then outputs a frame of audio data to the PING buffer, and so on in a loop.

[0030] Preferably, the data access module is built with a storage block mechanism, which divides the data access module into storage blocks, and each block stores a set of structured user data.

[0031] Preferably, in the data access module, each storage block includes:

[0032] head[4]: Used to describe what data this block is;

[0033] Version: The version of the structure of the currently stored data;

[0034] payload_array: The starting address of the user data structure to be written to the file, or it can also be an array of structures;

[0035] item_size: The number of bytes of a single structure;

[0036] array_total_size: The total size of the data of a single structure or an array of structures;

[0037] default_setting: Points to the default setting value, which can automatically complete initialization when the data stored by the user does not exist;

[0038] Pos: When it is an array of structures, it is to initialize one or more data using Default_setting;

[0039] Callback: A version-compatible callback function. When the version of the structure requested by the program is inconsistent with the version of the structure read out, the callback function is used to convert the read-out data according to different structure versions.

[0040] Preferably, for the MIDI command sending module, if the data frame does not require an answer, it is directly written to the sending circular buffer and the serial port transceiver module thread or the USB transceiver module thread is activated; if the data frame requires an answer, it is necessary to wait whether the previous instruction that requires an answer has been completed. Only when the waiting status is empty can this message be sent and set to the waiting-for-answer status, and the answer status is automatically cleared when the answer instruction is received.

[0041] The beneficial effects of the present invention are as follows:

[0042] The present invention can enable application engineers to focus on the development of product functions, reduce the energy input in building the system framework, greatly shorten the product development cycle, and improve the design efficiency.

[0043] 1. At the same time, the framework logic of the present invention is excellent, and the system is more stable, greatly reducing the program exception problems caused by personal abilities or incomplete considerations when application engineers develop their respective frameworks for different products, greatly reducing the investment in testing costs, and reducing the risks of product repair and customer complaints. Description of the Drawings

[0044] Figure 1 It is a schematic diagram of the overall architecture of a general musical instrument audio data processing system for supporting the MIDI protocol proposed by the present invention;

[0045] Figure 2 It is a schematic diagram of the write lock mechanism logic of a general musical instrument audio data processing system for supporting the MIDI protocol proposed by the present invention;

[0046] Figure 3 It is a schematic diagram of the standard structure in the serial port transceiver module of a general musical instrument audio data processing system for supporting the MIDI protocol proposed by the present invention;

[0047] Figure 4 It is a schematic diagram of the standard structure logic of a general musical instrument audio data processing system for supporting the MIDI protocol proposed by the present invention;

[0048] Figure 5 It is a schematic diagram of the USB data packing format of a general musical instrument audio data processing system for supporting the MIDI protocol proposed by the present invention;

[0049] Figure 6 It is a schematic diagram of the specific command codes and parameters in the USB data of a general musical instrument audio data processing system for supporting the MIDI protocol proposed by the present invention;

[0050] Figure 7 It is a schematic diagram of the transceiver logic of the USB transceiver module of a general musical instrument audio data processing system for supporting the MIDI protocol proposed by the present invention;

[0051] Figure 8 It is a schematic diagram of identifying the instruction type and its parameters based on the system code type of a general musical instrument audio data processing system for supporting the MIDI protocol proposed by the present invention;

[0052] Figure 9 It is a schematic diagram of the logic of the SYSEX MIDI command processing module of a general musical instrument audio data processing system for supporting the MIDI protocol proposed by the present invention;

[0053] Figure 10 It is a schematic diagram that application engineers of a general musical instrument audio data processing system for supporting the MIDI protocol can quickly add instructions to be supported according to the functional requirements of the product;

[0054] Figure 11 Schematic diagram of the APP / host computer communication protocol for a general musical instrument audio data processing system for supporting MIDI protocol proposed by the present invention;

[0055] Figure 12 Schematic diagram of the Bluetooth module communication protocol for a general musical instrument audio data processing system for supporting MIDI protocol proposed by the present invention;

[0056] Figure 13 Schematic diagram of the logic of the sound generation engine module for a general musical instrument audio data processing system for supporting MIDI protocol proposed by the present invention;

[0057] Figure 14 Schematic diagram of the storage block mechanism architecture for a general musical instrument audio data processing system for supporting MIDI protocol proposed by the present invention;

[0058] Figure 15 Schematic diagram of the user data storage list array for a general musical instrument audio data processing system for supporting MIDI protocol proposed by the present invention;

[0059] Figure 16 Schematic diagram of the debug commands for a general musical instrument audio data processing system for supporting MIDI protocol proposed by the present invention;

[0060] Figure 17 Schematic diagram of using a structure in a general musical instrument audio data processing system for supporting MIDI protocol to store the data after packet assembly;

[0061] Figure 18 Schematic diagram of the basic packet assembly process for a general musical instrument audio data processing system for supporting MIDI protocol proposed by the present invention;

[0062] Figure 19 Schematic diagram of the process of the MIDI command sending module for a general musical instrument audio data processing system for supporting MIDI protocol proposed by the present invention. Specific embodiments

[0063] The technical solutions of the present invention will be further described in detail below in conjunction with specific embodiments.

[0064] In the description of the present invention, it should be noted that unless otherwise clearly specified and limited, the terms "installation", "connection", "connection", "setting" should be understood in a broad sense. For example, it can be fixedly connected and set, or detachably connected and set, or integrally connected and set. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0065] Example 1

[0066] As Figure 1 shown, the present invention discloses a general musical instrument audio data processing system for supporting the MIDI protocol, including:

[0067] A circular buffer module 1, which is used to cache data from interrupts or another thread and serves as the main data interaction carrier between threads.

[0068] The circular buffer module 1 is built-in with a read semaphore mechanism and a write lock mechanism;

[0069] The logic of the read semaphore mechanism is as follows: when thread A reads the circular buffer, if the buffer data is empty or the amount of data requested is not enough, thread A will be suspended and the CPU resources will be released; until the buffer reaches the requested amount of data or the timeout, thread A will be reactivated;

[0070] The logic of the write lock mechanism is as follows: as Figure 2 shown, when thread B writes to the circular buffer, it will lock the buffer. If thread C also operates on the same buffer at this time, it will be suspended until thread B releases the lock, and then thread C will be activated.

[0071] Compared with the prior art, the circular buffer module of the present invention adds the above read semaphore mechanism and write lock mechanism for the system characteristics of multi-line systems.

[0072] The general musical instrument audio data processing system for supporting the MIDI protocol disclosed by the present invention further includes:

[0073] A serial port transceiver module 2, which receives data from the serial port bus through the DMA method and writes it into the receive circular buffer, activates the thread of the serial port transceiver module to call the MIDI stream decoding module through the semaphore; reads the data in the send circular buffer and sends it to the serial port bus through the DMA method.

[0074] As Figure 3 shown, in order to facilitate transplantation to different chip platforms, this module provides a standard structure.

[0075] Figure 4 Illustrates the process of using the standard structure for serial port initialization, circular buffer creation, and binding. The specific process is as follows:

[0076] The thread corresponding to the serial port transceiver module 2 starts to execute. First, it applies for the structure space of uart_dev_t, applies for the DMA space for receiving data and the DMA space for sending data, and registers them in the uart_dev_t structure, and sets parameters such as the DMA channel signal trigger source and the serial port baud rate to be used;

[0077] Initialize the serial port peripheral according to the uart_dev_t structure parameters;

[0078] Apply for the data reception circular buffer space according to rx_fifo_buff_len in uart_dev_t and apply for the data transmission circular buffer space according to tx_fifo_buff_len, and register them into the uart_dev_t structure;

[0079] After binding the serial port peripheral to the DMA channel, start the serial port, the DMA starts to wait for receiving data, and the thread enters the suspended state, waiting for serial port data;

[0080] When a DMA receive data interrupt occurs, the interrupt handling will write the received data into the receive circular buffer and send a semaphore to activate the thread. After the thread is activated, it will read the data in the receive circular buffer and call the MIDI stream decoding module to process this data;

[0081] When the MIDI command sending module enters data into the send circular buffer, it will synchronously send a semaphore to activate the thread. After the thread is activated, it will read the data in the send circular buffer and send the data to the serial port bus through the DMA method.

[0082] The general musical instrument audio data processing system for supporting the MIDI protocol disclosed by the present invention further includes:

[0083] A USB transceiver module 3, which receives or sends MIDI stream data through the USB bus.

[0084] Figure 5 This is a schematic diagram of the composition of a USB data for a general musical instrument audio data processing system for supporting the MIDI protocol proposed by the present invention. It can be seen that Byte0 in the USB data is the data type. As Figure 5 shown, according to the "USBMIDI V1.0" specification, each USB data is composed of 4 bytes. Among them, the data type (Cable Number) determined by Byte 0 has the valid length of the data (Code Index Number (CIN)). Byte1-3 are the valid data.

[0085] Figure 6 This is a schematic diagram of the specific command codes and parameters in the USB data of a general musical instrument audio data processing system for supporting the MIDI protocol proposed by the present invention;

[0086] Among them, all MIDI stream packetizing command codes (CIN) supported by "USB MIDI V1.0", and the valid data length (MIDI_x Size) corresponding to each command code.

[0087] As shown in Figure 7 , the receiving and transmitting processes of the USB MIDI stream by the USB transceiver module are illustrated:

[0088] The receiving process of the USB MIDI stream by the USB transceiver module is as follows: The USB triggers an interrupt, and the interrupt program reads 4 bytes of data from the receiving endpoint, determines the subsequent valid data length by judging the CIN code of Bit0...3 of Byte0, writes the valid data into the receiving circular buffer, and activates the USB transceiver module thread;

[0089] The transmitting process of the USB MIDI stream by the USB transceiver module is as follows: When the host initiates a data query instruction interrupt, the interrupt program reads up to 64 bytes of data from the transmitting circular buffer and sends it to the USB bus, that is, the data to be transmitted has been packetized by the MIDI command transmitting module according to the USB rule of 4 bytes.

[0090] The general musical instrument audio data processing system for supporting the MIDI protocol disclosed in the present invention further includes:

[0091] A MIDI stream decoding module 4. The list of MIDI command codes supported by the standard MIDI protocol "MIDI 1.0 Detailed Specification" is as Figure 8 shown. Decode the MIDI data stream and identify the instruction type and its parameters according to the system code type.

[0092] Figure 8 It is a schematic diagram for identifying the instruction type and its parameters of the system code type of the general musical instrument audio data processing system proposed by the present invention.

[0093] As Figure 8 shown, in the general musical instrument audio data processing system of the present invention, when identifying the instruction type and its parameters of the system code type, taking the most common command codes in the MIDI stream as examples are as follows:

[0094] ● Data stream of key press event: 0x91 0x3C 0x70. Among them, 0x91 indicates that a key press event has occurred on MIDI channel 1, 0x3C indicates that the pressed key is the key corresponding to the C4 note, and 0x70 indicates the intensity of the key press;

[0095] ● Data stream of key release event: 0x81 0x3C 0x20. Among them, 0x81 indicates that a key release event has occurred on MIDI channel 1, 0x3C indicates that the released key is the key corresponding to the C4 note, and 0x20 indicates the intensity of the key release;

[0096] ● Data stream for timbre change: 0xC0 0x02. Here, 0xC0 indicates that the timbre of MIDI channel 0 needs to be changed, and 0x02 indicates that the timbre is to be changed to that of an electric grand piano (assuming the user is currently using the standard GM timbre package);

[0097] ● Data stream for system extension instructions: 0xF0 0x00 0x60 0x50 0x00 0x00 0x0D 0x00 0x01 0xF7. Here, 0xF0 indicates the start of the system extension instruction, 0x00 0x60 0x50 0x00 0x00 0x0D 0x00 0x01 is the data of the user - private communication protocol, and 0xF7 indicates the end of the system extension instruction.

[0098] Figure 9 It is a logic schematic diagram of the SYSEX MIDI command processing module for a general - purpose musical instrument audio data processing system for supporting the MIDI protocol proposed by the present invention.

[0099] According to the MIDI protocol, each MIDI command consists of a command code and several parameters, and the value of the MIDI command code must be greater than 0x80. If the value of the command code is 0xF0, it means that the data received next is a variable - length extension command (SYSEX command), and the extension command ends until 0xF7 is received. Other values are ordinary MIDI commands. If it is an ordinary MIDI command, the ordinary MIDI command processing module (5) is triggered for invocation. If it is a SYSEX extension command, the SYSEX MIDI command processing module (6) is called. The specific processing flow is as Figure 9 shown.

[0100] The general - purpose musical instrument audio data processing system for supporting the MIDI protocol disclosed by the present invention further includes:

[0101] An ordinary MIDI command processing module 5, which processes ordinary MIDI commands, is used to control the sound - generating engine module to emit specified instrument sounds (such as: Note On code) or perform some effect (such as: CC code) processing, etc. At the same time, it sends the MIDI command data to the service processing module (12) to execute other user - defined services (such as: MIDI recording).

[0102] This module provides a callback function registration array outside, as Figure 10 shown. Application engineers can quickly add the instructions that need to be supported according to the functional requirements of the product.

[0103] In addition, this module has added a filter function, which allows users to decide which instructions need to be processed and which instructions do not need to be processed according to the product needs or user requirements, greatly increasing the convenience.

[0104] The functions for filter control provided to application engineers include:

[0105] void midi_channel_filter_add_or_rm(uint8_t midi_ch, uint16_t* filter, bool add): Add or remove the filter for the specified MIDI channel;

[0106] void midi_cc_filter_add_or_rm(uint8_t cc_code, uint8_t* filter, bool add): Add or remove the filter for the specified CC code;

[0107] void midi_code_filter_add_or_rm(uint8_t code, uint32_t* filter, bool add): Add or remove the filter for the specified command code;

[0108] Among them:

[0109] Midi_ch: Channel number. A MIDI device supports 16 channels, and each channel corresponds to an instrument;

[0110] Filter: The starting address of the filter array;

[0111] Add: Add / remove the specified cc_code to / from the filter. Among them:

[0112] true - Add the specified midi_ch to the filter;

[0113] Do not process the MIDI data stream of the specified channel;

[0114] false - Remove the specified midi_ch from the filter, and it is necessary to process the MIDI data stream of the specified channel to control the sound generation;

[0115] cc_code: Control code, used to control different sound generation effects, such as volume, pan, etc.;

[0116] Code: Sound generation command code, such as note on (key press), note off (key release), etc.;

[0117] The general musical instrument audio data processing system for supporting the MIDI protocol disclosed by the present invention further includes:

[0118] SYSEX MIDI command processing module 6, which processes the private communication protocol data, is used to control the working parameters of the device and handle the service interaction after the connection of the mobile phone APP.

[0119] The SYSEX MIDI command processing module 6 supports the APP / host computer communication protocol (as Figure 11 shown) and the Bluetooth module communication protocol (as Figure 12 shown).

[0120] Application engineers can support new communication protocols by simply changing the corresponding packet identification code arrays according to product requirements:

[0121] const uint8_t sysex_cmd_header[]: APP / host computer communication protocol identification code;

[0122] const uint8_t sysex_cmd_ble_module_header[]: Bluetooth module communication protocol identification code.

[0123] The general musical instrument audio data processing system for supporting the MIDI protocol disclosed by the present invention further includes:

[0124] A sound generation engine module 7, which adopts a PING-PONG double buffer mode and outputs the audio data in the PING buffer to the I2S peripheral through the DMA method.

[0125] Specifically, as Figure 13 shown, when the DMA completes a data transfer, an interrupt is triggered, and the audio data in the PONG buffer is output to the I2S peripheral through the DMA method. At the same time, an activation event is sent to the sound generation engine module thread to trigger the execution of a sound generation and effect processing, and then a frame of audio data is output to the PING buffer, and so on in a loop.

[0126] Application engineers only need to implement the corresponding sound generation algorithm and effect algorithm in this thread according to the product function requirements to generate a frame of audio data and fill the data into the corresponding buffer. By adopting the PING-PONG double buffer mode, the system can set different processing frame lengths according to the business complexity, ensuring that the user's sound generation algorithm can be processed in a timely manner and there will be no audio disconnection due to untimely processing caused by system jitter, interrupt preemption, system bus busyness, etc.

[0127] The general musical instrument audio data processing system for supporting the MIDI protocol disclosed by the present invention further includes:

[0128] A user data access module 8 for accessing and storing user data.

[0129] For the programs of general embedded products, there are two ways to store user data:

[0130] ● Method 1: When storing, directly serialize and write the user data (structure) into the memory. When reading, deserialize it into program variables. This method is difficult to expand user data. When new user settings are inserted, the original data will be lost.

[0131] ● Method 2: When storing, convert the user data into a formatted file such as XML / JSON, and then write it into the memory. When reading, convert the XML / JSON file back into program variables. This method can solve the problems of Method 1, but it requires more RAM space and higher chip computing power, making it difficult to implement on embedded chips.

[0132] To address the above problems, this module introduces a storage block mechanism. As Figure 14 shown, each block stores a set of structured user data. The block mechanism can solve the problem that the serialization method cannot expand new data, has the advantages of low computing power and high efficiency, and supports forward compatibility of user data. It also supports an automatic initialization function when the stored data does not contain the requested structure data, simplifying the entire data access process.

[0133] ◆ Each block contains:

[0134] head[4]: Used to describe what data this block is.

[0135] version: The version of the structure of the currently stored data

[0136] payload_array: The starting address of the user data structure to be written into the file, or it can be an array of structures

[0137] item_size: The number of bytes of a single structure

[0138] array_total_size: The total size of the data of a single structure or an array of structures

[0139] default_setting: Points to the default setting value. When the user's stored data does not exist, it can be automatically initialized

[0140] pos: When it is an array of structures, it indicates whether to initialize one or more data using Default_setting

[0141] callback: A callback function for version compatibility. When the version of the structure requested by the program is inconsistent with the version of the structure read out, this callback function will be called to convert the read data according to different structure versions.

[0142] ◆ As Figure 15As shown, the application engineer only needs to define a user data storage list array and pass it to user_data_write_to_file(array) to complete data storage. The application engineer only needs to call the user_data_read_fr_file(array) function to restore the read data to the corresponding structure variable.

[0143] The general musical instrument audio data processing system for supporting the MIDI protocol disclosed by the present invention further includes:

[0144] A character terminal module 9, which is used to receive input instructions from users, execute corresponding command processing after completing instruction interpretation, and output the system operation log to track the system operation status, facilitating application engineers to debug programs, verify module functions, track operation status, and solve program exceptions faster. The application engineer only needs to quickly add debug commands by calling the SHELL_EXPORT_FUNC command, such as Figure 12 as shown.

[0145] The general musical instrument audio data processing system for supporting the MIDI protocol disclosed by the present invention further includes:

[0146] A MIDI command packetizing module 10, which supports packetizing processing of ordinary MIDI commands, SYSEX MIDI commands, and data of the USB MIDI 4-byte protocol, and appends the packetized data frame to the send waiting queue and then activates the MIDI command sending module thread.

[0147] To simplify the programming complexity of application engineers, functions with different functions are provided:

[0148] ● For ordinary MIDI messages, this module provides the void

[0149] midi_code_send_frame_except_sysex(midi_send_ctx_idx_t send_ctx_idx,uint8_t*encoder_param,uint16_t len) function

[0150] ● For MIDI instructions used for communication with the host computer / APP, there are 4 types of messages:

[0151] ■ Communication type 1: Instructions sent to the host computer / APP but without requiring a response:

[0152] void midi_sysex_send_frame_without_ack(midi_send_ctx_idx_t send_ctx_idx, uint16_t cmd, uint8_t* param, uint16_t len)

[0153] ■ Communication type 2: Commands sent to the host computer / APP that require a response:

[0154] void midi_sysex_send_frame_with_ack(midi_send_ctx_idx_t send_ctx_idx, uint16_t cmd, uint8_t* param, uint16_t len, uint16_t wait_time)

[0155] ■ Communication type 3: Commands sent to the host computer / APP that require the requested data to be returned:

[0156] uint8_t midi_sysex_send_frame_with_rsp(midi_send_ctx_idx_t send_ctx_idx, uint16_t cmd, uint8_t* param, uint16_t len, uint16_t wait_time, uint8_t* rsp_buff, uint16_t* rsp_len)

[0157] ■ Communication type 4: Commands that return a response to the host computer / APP:

[0158] void midi_sysex_send_ack_frame(midi_send_ctx_idx_t send_ctx_idx, uint8_t org_cmd, uint8_t ack_sta, uint8_t* param, uint16_t len)

[0159] ● For MIDI commands used to communicate with the Bluetooth module, this module provides communication type 5 commands

[0160] void ble_model_cfg_send_frame_with_ack(uint8_t service_id, uint8_t cmd, uint8_t* param, uint16_t len, uint16_t wait_time)

[0161] For each command to be sent, such asFigure 17 As shown in the figure, this module designs the following midi_frame_t structure to store message data. The introduction of each element is as follows:

[0162] ◆ cmd: The command code corresponding to the message / instruction, stored in the function parameter encoder_param[0];

[0163] ◆ payload: Stores the message / instruction data to be sent to the physical transmission medium after packet assembly;

[0164] ◆ len: Records the length of the payload data;

[0165] ◆ sem: For instructions that require the host computer / APP to return requested data, the sending thread will enter a blocked state waiting for this semaphore;

[0166] ◆ wait_time: The maximum time for the slave computer to wait for an instruction response or for the requested data to be returned. If a timeout occurs, the message resending mechanism will be triggered;

[0167] ◆ rsp_buff: For instructions that require the host computer / APP to return requested data, this points to the memory address where the returned data is to be received;

[0168] ◆ rsp_buff_len: Records the capacity of rsp_buff. When the function returns, it will be changed to the actual size of the received data;

[0169] ◆ retry: Records the number of command resends. If no response or the requested data is not returned after exceeding the specified number of resends, the function will return a sending failure result.

[0170] The basic packet assembly process is as follows: When an application engineer calls any of the functions described above, the commands and parameters passed in will be packet-assembled for the array according to different communication types and interface methods inside the function, and then filled into the corresponding elements of the midi_frame_t structure variable. Finally, the structure is appended to the sending queue and the MIDI command sending module thread is activated to wait for transmission, as Figure 18 shown.

[0171] The general musical instrument audio data processing system for supporting the MIDI protocol disclosed in the present invention further includes:

[0172] The MIDI command sending module 11, which is responsible for managing the data frame sending queue and command response processing.

[0173] Read a data frame from the sending queue in a loop. If the type of the data frame is a normal MIDI message or communication type 1, directly write it into the sending circular buffer and activate the corresponding thread of the serial port transceiver module (2) or the corresponding thread of the USB transceiver module (3); if the type of the data frame is communication types 2 to 5, it is necessary to check whether the response waiting status is empty. Only when the waiting status is empty can this message be sent and the waiting for response status be set. When the response instruction is received, the response status is automatically cleared. For the specific process, refer to Figure 19 。

[0174] The general musical instrument audio data processing system for supporting the MIDI protocol disclosed by the present invention further includes:

[0175] The service processing module 12. After the application engineer completes the corresponding input / output detection, interaction logic, service functions, user settings, etc. according to the product functions, generates the corresponding normal MIDI codes and sends them to the sound generation engine module to output sounds, calls the MIDI command packaging module to send MIDI protocol commands to the serial port or USB or Bluetooth module, reads or saves user data to the storage medium through the user data access module, and receives debug instructions or outputs debug information through the character terminal module.

[0176] Embodiment 2

[0177] The present invention also discloses a general musical instrument audio data processing method for supporting the MIDI protocol, which is used to process the general musical instrument audio data from the user data access module (8). After the user data is read, the processing process of the entire platform software includes the following steps:

[0178] Step S1: After the system starts and runs, the user data access module (8) reads the user data and gives it to the service processing module (12). After the service module completes the initialization processing of the system according to these data, it enters the waiting state and is ready to receive message events from other modules;

[0179] Step S2: The character terminal module (9) receives the input instructions of the user, and after completing the instruction interpretation, executes the corresponding command processing;

[0180] Step S3: The serial port transceiver module (2) receives the data from the serial port bus through the DMA method and writes it into the receiving circular buffer, and activates the corresponding thread of the MIDI stream decoding module (3) through the semaphore; reads the data in the sending circular buffer and sends it to the serial port bus through the DMA method;

[0181] Step S4: The USB transceiver module (3) reads a data packet (4 bytes) from the receive endpoint in an interrupt manner through the USB bus, determines the valid data length of this packet according to the CIN code in Byte0, writes the valid data into the receive circular buffer, and activates the corresponding thread of the MIDI stream decoding module (3) through a semaphore; reads the data in the transmit circular buffer and sends it to the USB bus through the transmit endpoint;

[0182] Step S5: The MIDI stream decoding module (4) reads the data from the receive circular buffer, identifies the instruction type and the number of its parameters according to the system code type. If it is an ordinary MIDI command, it triggers the call of the ordinary MIDI command processing module (5). If it is a SYSEX extended command, it calls the SYSEX MIDI command processing module (6);

[0183] Step S6: The ordinary MIDI command processing module (5) executes the filter function to determine which instructions need to be processed and which do not; processes the MIDI commands, controls the sound engine module (7) to emit the specified instrument sound (e.g., Note On code) or execute some effects (e.g., CC code) processing, etc.; at the same time, sends the MIDI command data to the service processing module (12) to execute other user-defined services (e.g., MIDI recording);

[0184] Step S7: The sound engine module (7) adopts the PING-PONG double buffer mode to output the audio data in the PING buffer to the I2S peripheral through DMA. When DMA completes a data transfer, it triggers an interrupt, outputs the audio data in the PONG buffer to the I2S peripheral through DMA, and at the same time sends an activation event to the sound engine module thread to trigger the execution of a sound generation and effect processing, and then outputs one frame of audio data to the PING buffer, and so on in a loop;

[0185] Step S8: The SYSEX MIDI command processing module (6) processes the private communication protocol data, which is used to control the working parameters of the device and the service interaction processing after the connection of the mobile phone APP; supports the communication protocol instruction processing with the APP / host computer; supports the communication protocol instruction processing of the Bluetooth module; at the same time, transmits the instruction execution result to the service processing module (12) for further processing;

[0186] Step S9: After the service processing module (12) completes corresponding input / output detection, interaction logic, service functions, user settings, etc. according to the product functions, it generates corresponding ordinary MIDI codes and sends them to the sound generation engine module (7) to output sound; calls the MIDI command packaging module (10) to send MIDI protocol commands to the serial port, USB, or Bluetooth module; reads or saves user data to the storage medium through the user data access module (8); outputs debug information through the character terminal module (9).

[0187] Step S10: The MIDI command packaging module (10) performs packet processing on ordinary MIDI commands, SYSEX MIDI commands, and data groups of the USB MIDI 4-byte protocol, and appends the packetized data frames to the send waiting queue and then activates the MIDI command sending module thread (11).

[0188] Step S11: The MIDI command sending module (11) circularly reads a data frame from the send queue. If the type of the data frame is an ordinary MIDI message or communication type 1, it is directly written to the send circular buffer and the corresponding thread of the serial port transceiver module (2) or the corresponding thread of the USB transceiver module (3) is activated; if the type of the data frame is communication type 2-5, it is necessary to determine whether the response waiting status is empty. Only when the waiting status is empty can this message be sent and the waiting for response status be set. When the response instruction is received, the response status is automatically cleared.

[0189] The present invention designs a friendly human-computer interaction interface for the general musical instrument audio data processing system. At the same time, the framework logic of the present invention is excellent, the system is more stable, greatly reducing the program exception problems caused by application engineers developing their own frameworks for different products due to personal capabilities or incomplete considerations, greatly reducing the investment in testing costs, and reducing the risks such as product repair and customer complaints.

[0190] The present invention enables application engineers to focus on the development of product functions, reduces the energy investment in system framework construction, greatly shortens the product development cycle, and improves the design efficiency.

[0191] Those skilled in the art can clearly understand that the division of "units" or "modules" in the embodiments of the present invention is only a division of logical functions. In actual implementation, there may be other division methods. For example, multiple "units" or "modules" can be combined or integrated into one "unit" or "module" to achieve corresponding functions. Or one "unit" or "module" can be decomposed into multiple ones to jointly achieve corresponding functions. The "units" or "modules" in the embodiments of the present invention can be software and / or hardware that can independently complete or cooperate with other components to complete specific functions. The hardware can be, for example, FPGA (Field-Programmable Gate Array), IC (Integrated Circuit), etc., which will not be elaborated here one by one.

[0192] The embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the method in any of the foregoing embodiments are implemented. Among them, the computer-readable storage medium can include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nano-devices (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0193] The general musical instrument audio data processing system for supporting the MIDI protocol provided by the embodiments of the present invention may include a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the method in any of the foregoing embodiments are implemented.

[0194] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all of them should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A general musical instrument audio data processing system for supporting MIDI protocol, characterized in that: include: A serial port transceiver module (2) receives data from the serial port bus through DMA and writes it into a receiving ring buffer, and activates the thread of the serial port transceiver module through a semaphore to call the MIDI stream decoding module; Read the data in the send ring buffer and send it to the serial bus via DMA; A USB transceiver module (3) receives or sends MIDI stream data via a USB bus; A MIDI stream decoding module (4) decodes the MIDI data stream according to the standard MIDI protocol and identifies the instruction type and its parameters according to the system code type; A general MIDI command processing module (5) processes general MIDI commands to control the sound engine module to emit a specified instrument sound; SYSEX MIDI command processing module (6), which processes private communication protocol data and is used to control the working parameters of the device and the business interaction processing after the mobile phone APP is connected; The SYSEX MIDI command processing module (6) supports the APP / host computer communication protocol and the Bluetooth module communication protocol; The MIDI command packaging module (10) supports the data packaging processing of common MIDI commands, SYSEX MIDI commands, and USB MIDI 4-byte protocols, and activates the MIDI command sending module thread after appending the packaged data frame to the sending waiting queue; The MIDI command sending module (11) is responsible for managing the data frame sending queue and command response processing.

2. A universal musical instrument audio data processing system for supporting MIDI protocol according to claim 1, characterized in that: The general musical instrument audio data processing system for supporting the MIDI protocol also includes: A ring buffer module (1), which is used to cache data from an interrupt or another thread, and serves as the main data exchange carrier between threads; The ring buffer module has a built-in read semaphore mechanism and a write lock mechanism; The logic of the read semaphore mechanism is as follows: when thread A reads the circular buffer, if the buffer data is empty or does not contain the requested amount of data, thread A will be suspended and CPU resources will be released; until the buffer reaches the requested amount of data or the timeout period, thread A will be reactivated; The logic of the write lock mechanism is: when thread B writes to the circular buffer, it will lock the buffer. If thread C also operates on the same buffer at this time, it will be suspended until thread B releases the lock, and then thread C will be activated.

3. A universal musical instrument audio data processing system for supporting MIDI protocol according to claim 1, characterized in that: The receiving process of the USB transceiver module is as follows: the USB triggers an interrupt, the interrupt program reads 4 bytes of data from the receiving endpoint, determines the CIN code of Bit0...3 of Byte0 to determine the length of subsequent valid data, writes the valid data into the receiving ring buffer and activates the USB transceiver module thread; The sending process of the USB transceiver module is as follows: when the host terminal initiates a data query instruction interrupt, the interrupt program reads a maximum of 64 bytes of data from the sending ring buffer and sends it to the USB bus.

4. A universal musical instrument audio data processing system for supporting MIDI protocol according to claim 1, characterized in that: In the MIDI stream decoding module (4): If it is a normal MIDI command, it will trigger the call of the normal MIDI command processing module; If it is a SYSEX extended instruction, the SYSEX MIDI command processing module will be called.

5. A universal musical instrument audio data processing system for supporting MIDI protocol according to claim 1, characterized in that: The general musical instrument audio data processing system for supporting the MIDI protocol also includes: The sound engine module (7) adopts a PING-PONG double buffer mode to output the audio data in the PING buffer to the I2S peripheral via DMA; The working logic of the sound engine module is: when DMA completes a data transmission, an interrupt is triggered, and the audio data in the PONG buffer is output to the I2S peripheral via DMA. At the same time, an activation event is sent to the sound engine module thread to trigger a sound and effect processing, and then a frame of audio data is output to the PING buffer, and the cycle continues.

6. A universal musical instrument audio data processing system for supporting MIDI protocol according to claim 1, characterized in that: The general musical instrument audio data processing system for supporting the MIDI protocol also includes: A user data access module (8), used for accessing user data; The user data access module (8) has a built-in storage block mechanism, which divides the data access module into storage blocks, each block storing a set of structured user data. In the user data access module (8), each storage block includes: head[4]: used to describe the data type of this block; Version: the structure version of the current stored data; payload_array: the starting address of the user data structure to be written to the file, or it can be a structure array; item_size: the number of bytes of a single structure; array_total_size: the total data size of a single structure or structure array; default_setting: points to the default setting value. When the user's stored data does not exist, initialization can be completed automatically. Pos: When it is a structure array, one or more data are initialized using Default_setting; Callback: A version-compatible callback function. When the version of the structure requested by the program is inconsistent with the structure version read out, the callback function is used to convert the read data according to different structure versions.

7. A universal musical instrument audio data processing system for supporting MIDI protocol according to claim 1, characterized in that: The general musical instrument audio data processing system for supporting the MIDI protocol also includes: The character terminal module (9) is used to receive input instructions from the user and execute corresponding command processing after completing instruction interpretation.

8. A universal musical instrument audio data processing system for supporting MIDI protocol according to claim 1, characterized in that: The MIDI command sending module (11) directly writes the data frame into the sending ring buffer and activates the serial port transceiver module thread or the USB transceiver module thread if the data frame does not require a response; if the data frame requires a response, it needs to wait whether the previous instruction requiring a response has been completed, and only when the waiting state is empty can the message be sent and the waiting state be set, and the response state is automatically cleared when the response instruction is received.

9. A universal musical instrument audio data processing system for supporting MIDI protocol according to claim 1, characterized in that: The general musical instrument audio data processing system for supporting the MIDI protocol also includes: The business processing module (12) is a module where application engineers complete the corresponding input and output detection, interaction logic, business functions, user settings and other processing according to the product functions, generate corresponding common MIDI codes and send them to the sound engine module to output the sound, call the MIDI command packaging module to send MIDI protocol commands to the serial port or USB or Bluetooth module, read or save user data to the storage medium through the user data access module, and receive debugging instructions or output debugging information through the character terminal module.

Images