Audio simulation co-processing unit and audio protocol simulation method

Abstract

The present invention relates to an audio simulation coprocessing unit and an audio protocol simulation method. The method is implemented based on the audio simulation coprocessing unit and includes: a receiving simulation, wherein a sampling input subunit receives input audio data from an audio chip when a synchronous clock signal is at a falling edge, and transmits the data to an input FIFO subunit for caching; an input processing subunit reads the input audio data cached in the input FIFO subunit, performs preprocessing, and stores the data in a CPU; and a sending simulation, wherein an output processing subunit controls DMA to transfer output audio data from the CPU to an output FIFO subunit for caching; the output FIFO subunit writes the output audio data to a shift output subunit when a synchronous clock signal is at a rising edge; and the shift output subunit sends the written output audio data to the audio chip in a shifted manner. The present invention can support I2S and TDM audio protocols using pure software simulation, and the coprocessing unit can independently process audio data decoding, reducing CPU load.

Description

Technical Field

[0001] The present invention relates to the field of audio technology, and in particular to an audio simulation co-processing unit and an audio protocol simulation method. Background Art

[0002] With the rise of IoT devices, smart audio terminals, and in-vehicle multimedia applications, support for digital audio interfaces within MCUs (microcontroller units) has become increasingly important. In consumer electronics products such as smart speakers, headphones, and recording devices, MCUs must exchange high-speed digital audio data with audio codecs and converter chips (DACs or ADCs) using audio protocols to provide high-fidelity sound output or input. In automotive electronics, digital audio buses are used for data transmission within in-car entertainment systems, multi-channel speakers, and microphone arrays. Furthermore, in industrial control and IoT scenarios, voice interaction, audio alarms, or multi-channel sensor data acquisition may also require MCUs with corresponding audio interfaces. Therefore, integrating support for diverse audio protocols within MCUs is essential. This ensures that the MCU can directly connect to a variety of digital audio devices and meet the real-time and quality requirements for audio data transmission in diverse scenarios.

[0003] The most common audio protocols are I2S (Inter-IC Sound, a serial digital audio bus standard proposed by Philips) and TDM (Time-Division Multiplexing, a time-division multiplexing audio interface that can transmit multiple channel data at a time).

[0004] I2S is widely used for inter-chip dual-channel audio data transmission. Many current audio chips and MCUs support the I2S interface, making it the most widely used dual-channel digital audio transmission protocol for a variety of consumer electronics and embedded systems. The I2S bus typically consists of a clock line (BCLK), left and right channel select lines (LRCLK or WS), and an audio data line (SD), enabling synchronous transmission of high-quality stereo audio data. However, I2S itself only supports two audio channels, which limits its use in practical multi-channel audio applications.

[0005] To accommodate more channels, the industry introduced TDM, which can be considered an extended variant of the I2S audio protocol—I2S is essentially a form of "TDM2" (two-channel TDM). TDM and I2S share essentially the same hardware connections and pin definitions, but utilize a higher bit clock frequency and frame synchronization signal to transmit multiple channels of data within a single audio frame. The audio sampling frequency is typically 44.8 kHz, but for a TDM multi-channel interface, the data transmission rate can reach 12.288 MHz. The audio protocol type and input and output frequencies can be modified through configuration to accommodate a variety of audio sampling or decoding chips.

[0006] Currently, there are two methods for integrating I2S / TDM interfaces into MCUs. One is hardware integration. However, integrating hardware interfaces increases circuit size, occupies additional chip area, and increases chip manufacturing costs. IP acquisition incurs licensing fees, and the addition of additional IP requires a complex process, including system integration testing and respins. Furthermore, the number of integrated I2S / TDM interfaces is limited. If an application requires more audio channels or additional audio interfaces than the chip's native interface capacity, existing hardware may not be able to support them, necessitating the use of a higher-end MCU or the addition of an external converter chip.

[0007] Another approach is to use pure software combined with GPIO (General Purpose Input Output) / SPI (Serial Peripheral Interface) / UART (Universal Asynchronous Receiver / Transmitter) to simulate I2S / TDM timing. The GPIO approach requires the MCU to flip the GPIO and move data, which takes up too much load and makes it almost impossible to handle other tasks normally. The SPI / UART approach uses DMA to handle data movement tasks, and using SPI / UART to receive or send data can reduce the CPU load. However, the speeds of these two IPs can only support the lower-frequency I2S audio protocol and cannot support the TDM audio protocol (i.e., the higher-frequency I2S audio protocol).

[0008] Furthermore, a patent has been published (CN105389155A), which proposes "a method and system for receiving TDM audio data using an SPI interface". It uses an SPI (short for Serial Peripheral Interface, a synchronous peripheral interface that enables a single-chip microcomputer to communicate with various peripheral devices in a serial manner to exchange information) interface to receive TDM audio data sent by the host in slave DMA (short for Direct Memory Access, which refers to direct memory access) mode. The frame synchronization signal of TDM is connected to the slave chip select signal of SPI, the clock signal of TDM is connected to the bit clock signal of SPI, and the data signal of TDM is connected to the host output slave input signal of SPI. The information received from the SPI data register is transferred to the CPU memory via DMA, and the CPU is subsequently used for data processing. Although this method supports the TDM audio protocol through pure software simulation. However, this method has the following disadvantages: the SPI frame length does not match the TDM frame length, requiring the CPU to participate in splicing the acquired data, which increases the CPU processing complexity; the SPI transmission rate is limited, and the sampling of high-speed audio signals may cause noise or data loss; SPI can only simulate the TDM audio protocol as a receiving end and cannot simulate the sending of TDM data. Summary of the Invention

[0009] In order to solve the above problems, the present invention provides an audio simulation co-processing unit and an audio protocol simulation method, which can support I2S and TDM audio protocols in a pure software simulation manner, and the co-processing unit can independently process audio data decoding, reducing the CPU load.

[0010] The present invention is implemented through the following scheme: an audio simulation co-processing unit is arranged between the CPU and the audio chip, and the audio simulation co-processing unit includes:

[0011] an input / output interface unit, comprising a sampling input subunit for receiving input audio data from the audio chip and storing the data in the CPU, a shift output subunit for sending output audio data from the CPU to the audio chip in a shifted manner, and a clock control subunit for controlling the sampling input subunit and the shift output subunit by generating a synchronous clock signal, wherein the shift output subunit is configured to be rising-edge valid and the sampling input subunit is configured to be falling-edge valid;

[0012] a FIFO data buffer unit, comprising an input FIFO subunit for buffering the input audio data and an output FIFO subunit for buffering the output audio data, wherein the input FIFO subunit is connected between the sampling input subunit and the CPU, and the output FIFO subunit is connected between the CPU and the shift output subunit;

[0013] The data processing unit includes an output processing subunit for controlling DMA to transfer the output audio data from the CPU to the output FIFO subunit, and an input processing subunit for reading the input audio data in the input FIFO subunit and storing it in the CPU after pre-processing. The output processing subunit is controlled and connected to the DMA, the DMA is connected between the CPU and the output FIFO subunit, and the input processing subunit is connected between the input FIFO subunit and the CPU.

[0014] The audio simulation co-processing unit of the present invention is further improved in that:

[0015] The amount of data transferred each time by the DMA, the capacity of the output FIFO subunit and the input FIFO subunit are all adapted to the size of one frame of TDM audio data, and the output FIFO subunit is provided with a low water mark for indicating that the output FIFO subunit is empty, and the input FIFO subunit is provided with a high water mark for indicating that the input FIFO subunit is full;

[0016] The output processing subunit is configured to control the DMA to start transferring output audio data only when the low water mark is detected, and the input processing subunit is configured to read input audio data of the input FIFO subunit only when the high water mark is detected.

[0017] A further improvement of the audio simulation co-processing unit of the present invention is that two micro-kernels that can run in parallel are provided in the input processing unit, one is used to cyclically monitor the input FIFO sub-unit and read the input audio data of the input FIFO sub-unit when the high water mark is detected, and the other is used to pre-process the read input audio data and store it in the input processing sub-unit of the CPU.

[0018] A further improvement of the audio simulation co-processing unit of the present invention is that a filtering subunit is provided in the sampling input subunit for filtering the sampled input audio data before outputting it.

[0019] A further improvement of the audio simulation co-processing unit of the present invention is that the input-output interface unit also includes an input-output control subunit, and the input-output control subunit is designed with a start control signal based on the input-output frequency configuration, which is used to control the start and stop time of the shift output subunit sending the output audio data.

[0020] A further improvement of the audio simulation co-processing unit of the present invention is that the clock control subunit can support a clock output frequency of 12.288 MHz, the clock output frequency is generated based on the clock system frequency division, and the clock output frequency can be adjusted by modifying the frequency division coefficient.

[0021] A further improvement of the audio simulation co-processing unit of the present invention is that the number of bits used by the shift output subunit to shift the audio data is 16 bits.

[0022] The present invention also provides an audio protocol simulation method, which is implemented based on the audio simulation co-processing unit described above, and the audio protocol simulation method includes:

[0023] Receiving simulation, using the sampling input subunit to receive the input audio data of the audio chip when the synchronous clock signal is at the falling edge, and transmitting it to the input FIFO subunit for buffering. The input processing subunit reads the input audio data buffered in the input FIFO subunit, performs pre-processing, and stores it in the CPU;

[0024] The sending simulation is performed, and the output processing subunit controls the DMA to move the output audio data from the CPU to the output FIFO subunit for caching. The output FIFO subunit writes the output audio data to the shift output subunit when the synchronous clock signal is a rising edge. The shift output subunit sends the written output audio data to the audio chip in a shifted manner.

[0025] The present invention includes but is not limited to the following beneficial effects:

[0026] 1. The co-processing unit of the present invention can independently complete the transmission and reception simulation of audio data and the pre-processing of audio data. The entire transmission process does not require CPU participation, which solves the problems of excessive CPU load and transmission frequency limitation caused by software combining GPIO / SPI / UART to simulate I2S / TDM timing.

[0027] 2. The present invention supports I2S and TDM audio protocols by adopting a pure software simulation method, thereby solving the limited channel resources and cost problems caused by the hardware integration method.

[0028] 3. The sending and receiving channels of the co-processing unit of the present invention are expandable, and each channel can have a higher data transmission rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 The figure shows a principle block diagram of the audio simulation co-processing unit of the present invention. DETAILED DESCRIPTION

[0030] In order to address the defects of hardware integration and software combined with GPIO / SPI / UART simulation, the present invention provides an audio simulation co-processing unit and an audio protocol simulation method, which can support I2S and TDM audio protocols by pure software simulation. The co-processing unit can independently handle audio data decoding, reducing the CPU load.

[0031] The audio simulation co-processing unit and the audio protocol simulation method are further described below with reference to specific embodiments and accompanying drawings.

[0032] See Figure 1 As shown, an audio analog coprocessing unit is arranged between the CPU and the audio chip. The audio analog coprocessing unit includes an input / output interface unit, a FIFO data cache unit, and a data processing unit. The input / output interface unit is responsible for sampling the input audio data of the audio chip and storing it in the CPU's receive data register, and outputting the output audio data in the CPU's transmit data register by shifting. The FIFO data cache unit is responsible for caching the input and output audio data to ensure the continuity of the audio data. The data processing unit is responsible for controlling the DMA to send the output audio data to the FIFO data cache unit, as well as reading the information in the FIFO data cache unit and performing preprocessing.

[0033] Specifically, the input / output interface unit includes a sampling input subunit, a shift output subunit, and a clock control subunit. The sampling input subunit is used to receive the input audio data from the audio chip and store it in the receive data register (located in the data memory RAM). The shift output subunit is used to send the output audio data in the transmit data register (located in the data memory RAM) to the audio chip in a shifted manner. The clock control subunit generates a synchronous clock signal to control the sampling input subunit and the shift output subunit. The shift output subunit is configured to be valid on the rising edge, while the sampling input subunit is configured to be valid on the falling edge. In other words, the shift output subunit is controlled by the synchronous clock signal generated by the clock control subunit. Each rising edge of the clock signal triggers the start of the shift output, and the output stops until the next rising edge. The shift output subunit is provided with a shadow register and a data register. The shift output data is first stored in the shadow register. After the shift output is completed, the shadow register immediately updates the data register to ensure continuity between the two shifts. The sampling input subunit is controlled by the same synchronous clock signal. The falling edge of each clock signal triggers the start of receiving input audio data. A result register is provided in the sampling input subunit. The received input audio data is stored in the result register and is moved out when the result register is full. Preferably, the sampling input subunit is provided with a filtering subunit for filtering the sampled input audio data before outputting it. The filtering processing of the filtering subunit can ensure the signal quality and filter out the noise in the input audio data. Preferably, the audio analog co-processing unit is provided with two groups of input and output interface units to achieve the expansion of the transmission channel. If more audio interfaces are required, it can also be expanded to other sub-units with the same functional attributes in the co-processing unit.

[0034] The FIFO data cache unit includes an input FIFO subunit and an output FIFO subunit. The input FIFO subunit is used to cache the input audio data to ensure the continuity of the input audio data. The output FIFO subunit is used to cache the output audio data to ensure the continuity of the output audio data. The FIFO transmits the output audio data to the shadow register of the shift output subunit through a blocking write mode, ensuring that the data is provided immediately after the shift is completed and no data overwriting occurs. The input FIFO subunit is connected between the sampling input subunit and the CPU, and the output FIFO subunit is connected between the CPU and the shift output subunit.

[0035] The data processing unit includes an output processing subunit and an input processing subunit. The output processing subunit is used to control the DMA to transfer the output audio data from the CPU to the output FIFO subunit. The input processing subunit is used to read the input audio data from the input FIFO subunit and store it in the CPU after pre-processing. The output processing subunit is controlled and connected to the DMA, which is connected between the CPU and the output FIFO subunit. The input processing subunit is connected between the input FIFO subunit and the CPU. The data processing unit independently controls the timing of data transmission in the entire co-processing unit and can perform simple data processing, greatly reducing the CPU load and improving system operation efficiency.

[0036] As a preferred embodiment, the audio chip is equipped with an I2S / TDM interface. The I2S audio data output includes three channels: left and right channel selection lines, a clock line, and an audio data line. The I2S audio protocol only supports two channels, so the data line contains two 16-bit slots (frames). The TDM audio protocol extends the I2S audio protocol, converting the left and right channel signals into synchronization signals. The data signal supports 16 channels, so the data line contains 16 16-bit slots. The I2S sampling rate is 44.8 kHz, and the corresponding I2S data transmission clock frequency is 44.8 kHz * 2 * 16. In order for the audio analog co-processing unit to support both I2S and TDM audio protocols, TDM and I2S should have the same sampling rate of 44.8 kHz, that is, the TDM data transmission clock frequency needs to be increased to 44.8 kHz * 16 * 16 = 12.288 MHz. The shift output subunit and the sampling input subunit are respectively used to support I2S / TDM protocol output and input. The clock control subunit generates a synchronous clock signal based on the clock system frequency division. It can be provided to the shift output subunit and the sampling input subunit within the audio analog co-processing unit to control the transmission and reception of audio data. It can support a clock output frequency of 12.288MHz. By using the same clock signal control, the output synchronization of the audio synchronization signal / left and right channel signals and the data signal can be guaranteed. Preferably, the frequency division coefficient can be modified to adjust the clock output frequency.

[0037] Furthermore, in this embodiment, each time the DMA transfers one frame of TDM audio data, the capacity of the output FIFO subunit and the input FIFO subunit are adapted to (preferably slightly larger than) the size of one frame of TDM audio data, and the output FIFO subunit is provided with a low watermark for indicating that the output FIFO subunit is empty (or nearly empty), and the input FIFO subunit is provided with a high watermark for indicating that the input FIFO subunit is full (or nearly full). The output processing subunit is configured to control the DMA to begin transferring output audio data only upon detecting the low watermark, and the input processing subunit is configured to read the input audio data of the input FIFO subunit only upon detecting the high watermark. Accordingly, the number of bits by which the shift output subunit shifts the audio data is 16 bits, ensuring that a set of shift bits matches the length of a frame, thereby ensuring the flexibility of the output audio protocol.

[0038] As a preferred embodiment, to enable the audio simulation co-processing unit to simultaneously support I2S audio protocols and TDM audio protocols with different frequencies and configurations, the input / output interface unit also includes an input / output control subunit for controlling the switching between different audio protocols and different input / output frequency configurations. The shift output subunit needs to control the start and end times of data frame transmission. Therefore, the input / output control subunit is designed with a start control signal. Upon receiving the start control signal, the shift output subunit transmits a frame of data. After completing a frame of data transmission, it waits for the start control signal to arrive before starting to transmit the next frame.

[0039] As a preferred embodiment, the input processing unit is equipped with two microkernels that can run in parallel: one for cyclically monitoring the input FIFO subunit and reading the input audio data from the input FIFO subunit when the high water mark is detected, and the other for pre-processing the read input audio data and storing it in the CPU's input processing subunit. Correspondingly, the output processing unit is equipped with a microkernel that cyclically monitors the output FIFO subunit and quickly triggers DMA for data transfer when the low water mark is detected. Tasks of different microkernels will not be interrupted by each other. This ensures rapid data calculation and processing, preventing data loss due to data overflow in the FIFO data cache unit.

[0040] Based on the audio simulation co-processing unit described above, the present invention also provides an audio protocol simulation method, which includes receiving simulation and sending simulation, and the steps are as follows:

[0041] 1. Receiving simulation: The sampling input subunit receives the input audio data from the audio chip when the synchronous clock signal is at the falling edge, and transmits it to the input FIFO subunit for buffering. The input processing subunit reads the input audio data buffered in the input FIFO subunit, pre-processes it, and stores it in the CPU.

[0042] 2. Send simulation: the output processing subunit controls the DMA to move the output audio data from the CPU to the output FIFO subunit for caching. The output FIFO subunit writes the output audio data to the shift output subunit when the synchronous clock signal is a rising edge. The shift output subunit sends the written output audio data to the audio chip in a shifted manner.

[0043] The present invention supports I2S and TDM audio protocols, can adjust the data frequency, supports 12.288MHz clock frequency, and is flexible in application. In addition, the audio simulation co-processing unit is scalable. Usually, there are more demands for audio protocol channels in application scenarios, so an expandable input and output interface unit is designed. Each basic design can expand 2 groups of input and output interface units. If there are more audio interface requirements, it can be expanded to other sub-units with the same functional attributes in the co-processing unit. The present invention realizes high-speed data handling and storage through the design of FIFO data cache unit and DMA, ensuring continuous high-speed input and output of data of the input and output interface unit. The audio simulation co-processing unit of the present invention is independent of the CPU and can independently complete the sending and receiving simulation of audio data and the pre-processing of audio data. The entire data sending and receiving process does not require the participation of the CPU.

[0044] The present invention has been described in detail above with reference to the embodiments of the accompanying drawings. A person skilled in the art can make various modifications to the present invention based on the above description. Therefore, certain details in the embodiments should not be construed as limiting the present invention. The scope of protection of the present invention shall be determined by the scope defined in the appended claims.

Claims

1. An audio simulation co-processing unit, characterized in that: Set between the CPU and the audio chip, the audio simulation co-processing unit includes: an input / output interface unit, comprising a sampling input subunit for receiving input audio data from the audio chip and storing the data in the CPU, a shift output subunit for sending output audio data from the CPU to the audio chip in a shifted manner, and a clock control subunit for controlling the sampling input subunit and the shift output subunit by generating a synchronous clock signal, wherein the shift output subunit is configured to be rising-edge valid and the sampling input subunit is configured to be falling-edge valid; a FIFO data buffer unit, comprising an input FIFO subunit for buffering the input audio data and an output FIFO subunit for buffering the output audio data, wherein the input FIFO subunit is connected between the sampling input subunit and the CPU, and the output FIFO subunit is connected between the CPU and the shift output subunit; The data processing unit includes an output processing subunit for controlling DMA to transfer the output audio data from the CPU to the output FIFO subunit, and an input processing subunit for reading the input audio data from the input FIFO subunit and storing it in the CPU after pre-processing. The output processing subunit is controlled and connected to the DMA, the DMA is connected between the CPU and the output FIFO subunit, and the input processing subunit is connected between the input FIFO subunit and the CPU; wherein, The audio chip is provided with an I2S / TDM interface, the shift output subunit and the sampling input subunit are used to support I2S / TDM protocol output and input respectively, and the clock control subunit generates a synchronous clock signal based on the clock system frequency division, which can be provided to the shift output subunit and the sampling input subunit inside the audio analog coprocessing unit to control the sending and receiving of audio data, and can support a clock output frequency of 12.288MHz.

2. The audio simulation co-processing unit according to claim 1, wherein: The amount of data transferred each time by the DMA, the capacity of the output FIFO subunit and the input FIFO subunit are all adapted to the size of one frame of TDM audio data, and the output FIFO subunit is provided with a low water mark for indicating that the output FIFO subunit is empty, and the input FIFO subunit is provided with a high water mark for indicating that the input FIFO subunit is full; The output processing subunit is configured to control the DMA to start transferring output audio data only when the low water mark is detected, and the input processing subunit is configured to read input audio data of the input FIFO subunit only when the high water mark is detected.

3. The audio simulation co-processing unit according to claim 2, wherein: The input processing subunit is provided with two microkernels that can run in parallel, one for cyclically monitoring the input FIFO subunit and reading the input audio data of the input FIFO subunit when the high water mark is detected, and the other for pre-processing the read input audio data and storing it in the input processing subunit of the CPU.

4. The audio simulation co-processing unit according to claim 1, wherein: The sampling input subunit is provided with a filtering subunit for filtering the sampled input audio data before outputting it.

5. The audio simulation co-processing unit according to claim 1, wherein: The input / output interface unit further includes an input / output control subunit. The input / output control subunit is configured with a start control signal based on the input / output frequency configuration, and is used to control the start and stop time of the shift output subunit in sending the output audio data.

6. The audio simulation co-processing unit according to claim 1, wherein: The number of bits by which the shift output subunit shifts the audio data is 16 bits.

7. An audio protocol simulation method, characterized in that: Based on the audio simulation co-processing unit according to any one of claims 1 to 6, the audio protocol simulation method includes: Receiving simulation, using the sampling input subunit to receive the input audio data of the audio chip when the synchronous clock signal is at the falling edge, and transmitting it to the input FIFO subunit for buffering. The input processing subunit reads the input audio data buffered in the input FIFO subunit, performs pre-processing, and stores it in the CPU; The sending simulation is performed, and the output processing subunit controls the DMA to move the output audio data from the CPU to the output FIFO subunit for caching. The output FIFO subunit writes the output audio data to the shift output subunit when the synchronous clock signal is a rising edge. The shift output subunit sends the written output audio data to the audio chip in a shifted manner.

Images