Programmable chip-based usb microphone array real-time full data stream method

Abstract

The application discloses a USB microphone array real-time complete data stream method based on a programmable chip, which comprises a ping-pong buffer, a main loop decoding, a ring queue and a USB IN interrupt program module. A large data stream needs to be transmitted and processed in a microphone array, and the process needs to ensure the real-time and completeness of the data stream. To this end, a software ping-pong buffer is first designed, two equal-length buffer arrays are opened, and the PDM data stream continuously output by the microphone is alternately received; then, when a certain ping-pong buffer is full of data, the data is put into the main loop program in a block to extract and decode, and the PCM data block is obtained; finally, the PCM data block decoded each time is directly written into the designed ring queue, and when the USB IN interrupt occurs, the data is dynamically read from the ring queue and uploaded by using an adaptive algorithm. In addition, the size of the ping-pong buffer is designed to ensure the completeness of the data stream. The microphone array with the programmable chip as the core can ensure the real-time and completeness of the transmission and processing of the large data stream.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of electronics and communication, and more particularly relates to a USB microphone array real-time complete data stream method based on a programmable chip. BACKGROUND

[0002] A microphone array is the basis for realizing sound source positioning, intelligent video conferencing, intelligent sensing and other applications. A USB microphone array platform developed based on an embedded programmable chip is a convenient and low-cost solution, and has great commercial application value. The key to realizing the array function lies in the design of the firmware program of the programmable chip, and the core of the firmware program is the audio data stream architecture. In order to facilitate developers to develop USB audio devices based on the convenient open-source programmable chip, the existing data stream solution cannot guarantee the real-time and integrity of the data in the process of transmitting and processing the large audio data stream in the programmable chip with limited computing power. Therefore, for a microphone array with a programmable chip as the core, an audio data stream architecture method is needed, which is more efficient, detailed and can maximize the guarantee of the real-time and integrity of the data stream. The design of the audio data stream needs to consider the following key problems:

[0003] 1) Audio data stream transmission lag and frame loss

[0004] Since the firmware program needs to read, write, transmit and process the large audio data stream in real time, the program task in the host chip is busy. If the program is blocked and the execution process is time-consuming, and the data is lost in the read-write process, the real-time audio data stream will be damaged and the effective data will be lost.

[0005] 2) Asynchronous clock transmission of USB

[0006] Since there is no synchronous clock line set on the USB bus, the USB bottom layer protocol does not provide an inherent clock synchronization mechanism, so the communication between the USB microphone array and the system platform is asynchronous clock communication, and the audio data stream transmission process will have the problem of asynchronous read and write across clock domains. With the increase of working time, the data overflow or read empty phenomenon in the cache area will occur, that is, the effective data will be lost. SUMMARY

[0007] In order to overcome the shortcomings of the prior art, for a USB microphone array with an embedded programmable chip as the control core, the internal audio data stream architecture thereof is designed and optimized in detail, and the purpose is to provide an audio data stream architecture method which is more efficient, detailed and can maximize the guarantee of the real-time and integrity of the data stream.

[0008] In order to achieve the above purpose, the technical scheme adopted by the present application is:

[0009] The USB microphone array real-time complete data stream method based on programmable chip comprises:

[0010] The ping-pong buffer module is used for providing efficient temporary buffer area to ensure the real-time performance of receiving and subsequent extraction and decoding operation of the microphone PDM data stream, and the main technical method is that two equal-length buffer arrays buffer0 and bufferl are opened, and the two buffers are alternately received the PDM data stream continuously output by the microphone in the DMA transmission process.

[0011] The main loop decoding module is used for extracting and filtering the PDM data stream output by the microphone on the software to decode into the required PCM data stream, and the process consumes CPU resources, and the main technical method is that when it is determined that a certain buffer array in the ping-pong buffer is full of PDM data, the main loop extracts and decodes the buffer array in blocks to further decode into a PCM data block.

[0012] The ring queue module is used for providing sufficient memory space for fast data reading and writing to ensure the real-time performance of the data stream, and also needs to ensure the integrity of the data stream, that is, no frame loss, and the main method is that a one-dimensional array que[M] with M=2N (N is a positive integer) elements is opened in the MCU memory, and the inversion operation of the read / write pointer (that is, the index value of the array element, which is represented by wr_ptr / rd_ptr) is realized by using the bit AND operation.

[0013] The USB IN interrupt module mainly adopts an adaptive algorithm to dynamically read an appropriate amount of audio data from the ring queue to stabilize the data amount in the device, and upload the data when the system platform requests the data.

[0014] Preferably, the ping-pong buffer module and the main loop decoding module need to start the DMA reception interrupt mechanism in the MCU to complete the operation, for example, at the first time, the DMA reception task and the DMA reception completion interrupt are started, the target buffer of the DMA transmission PDM data stream is buffer0, and the interrupt is triggered when buffer0 is full; then, in the interrupt processing function, the target buffer of the DMA reception is changed to bufferl, and then the full flag of buffer0 is set; at the same time, the full flags of buffer0 and bufferl are determined in the main loop, and if the determination is true, the corresponding buffer array is extracted and decoded in blocks on the software.

[0015] Preferably, the implementation method of the ring queue module is as follows:

[0016] 1) The following operations are used to realize the fast writing of PCM data into the ring queue:

[0017] que[wr_ptr] = wr_value

[0018] wr_ptr = (wr_ptr++) & (M-1)

[0019] wherein, when wr_ptr increases to an integer multiple of M, wr_ptr = 0 is set, i.e. the reversal of the write pointer wr_ptr is completed;

[0020] 2) The following operations are respectively used to realize fast reading of PCM data from the ring queue for USB uploading:

[0021] rd_value = que[rd_ptr]

[0022] rd_ptr = (rd_ptr++) & (M-1)

[0023] wherein, when rd_ptr increases to an integer multiple of M, rd_ptr = 0 is set, i.e. the reversal of the read pointer rd_ptr is completed.

[0024] Preferably, for the PCM data block decoded by the main loop decoding program module, the mode of direct writing into the ring queue is adopted in the ring queue program module, and in order to avoid the problem of frame loss caused by the out-of-bound of the write data pointer when the tail is adjacent, the following method is used:

[0025] The size L of the two segments of the ping-pong buffer, buffer0 and bufferl, is set to 2 k words, and the length m of each segment of the buffer after extraction and decoding into a PCM data block is 2 k-1 words (each 32-bit PDM data block is converted into 16-bit 1 frame), and thus the length M of the ring queue is an integer multiple of the length m of the PCM data block, avoiding the phenomenon of intra-block data frame loss when the ring queue is full after direct writing of the data block.

[0026] Preferably, the main loop decoding program module writes the decoded m = 2 k-1 word PCM data block into the ring queue, and updates the write pointer wrptr + = m of the ring queue.

[0027] Preferably, the USB IN interrupt program module uses the IN endpoint, and in order to maintain the stable amount of device data, an adaptive algorithm is used to dynamically read an appropriate amount of data from the ring queue for uploading to the system. The main method is as follows: first, the amount of data that has been collected but not uploaded is accurately calculated, then the adjustment amount is calculated by comparing the amount of data with the amount of data in the ring queue when it is about half full, and finally the adjustment amount is used to dynamically increase or decrease the amount of data for uploading, so that the amount of data that has been collected but not uploaded is maintained at about half full of the ring queue. The specific method is as follows:

[0028] 1) The method for accurately calculating the current collected and not uploaded data amount remain len is:

[0029] remain len = (wr_ptr - rd_ptr) + (L - remain_DMA_len) / 2

[0030] In the formula, remain_DMA_len represents the remaining data amount not received by DMA, and the unit is words.

[0031] 2) For the calculation of the adjustment amount, the difference (remain_len - M / 2) of remain_len and the semi-full data amount of the ring queue is scaled by 8 times as the adjustment amount diff, that is,

[0032] diff = (remain_len - M / 2) / 8

[0033] The above method reduces data fluctuations and gradually synchronizes the data amount adjustment.

[0034] 3) The uploaded data amount sendlen is determined by the adjustment amount. Based on the original 96 uploaded data amount, if the difference is greater than 0, the uploaded data amount is sendlen = 96 + diff; if the difference is less than 0, the uploaded data amount is sendlen = 96 - diff; wherein the maximum adjustment amount needs to be limited, and when the calculated adjustment amount diff is greater than 24, only 24 is taken, that is, the range of sendlen is between 72 and 120 words.

[0035] The present application proposes an audio data stream architecture method which is more efficient, detailed, and can maximize the guarantee of real-time and integrity of data stream, can be used in the microphone array firmware developed based on programmable chips, so that the array device has good performance. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is the overall architecture diagram of the present application.

[0037] Figure 2 is the main loop decoding program module of the present application.

[0038] Figure 3 is the ring queue data structure and data read / write schematic diagram of the present application.

[0039] Figure 4 is the schematic diagram of the present application showing the out-of-bound frame loss phenomenon generated by direct data writing.

[0040] Figure 5 is the calculation schematic diagram of the adaptive algorithm dynamically reading the appropriate amount of data from the ring queue of the present application. DETAILED DESCRIPTION

[0041] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and examples.

[0042] The USB microphone array real-time complete data stream method based on programmable chip has a whole structure as shown in Figure 1 The USB microphone array real-time complete data stream method based on programmable chip has a whole structure as shown in

[0043] The ping-pong buffer module is used to provide efficient temporary buffer area to ensure the real-time performance of receiving the microphone PDM data stream and subsequent extraction and decoding operation. The main technical method is to open two equal-length buffer arrays buffer0 and buffer1, and in the DMA transmission process, the two buffers are alternately received the PDM data stream continuously output by the microphone.

[0044] The main loop decoding module, as shown in Figure 2 is used to extract and filter the PDM data stream output by the microphone on the software to decode into the required PCM data stream. The process consumes CPU resources. The main technical method is that when it is determined that a certain buffer array in the ping-pong buffer is full of PDM data, the main loop extracts and decodes the buffer array in blocks to further decode into a PCM data block.

[0045] The ring queue module, as shown in Figure 3 is used to provide sufficient memory space for fast data read and write to ensure the real-time performance of the data stream, and also needs to ensure the integrity of the data stream, that is, no frame loss. The main method is to open a one-dimensional array que[M] with M=2 N (N is a positive integer) in the MCU memory, and use bit operation to realize the inversion operation of the read / write pointer (that is, the index value of the array element, which is represented by wr_ptr / rd_ptr respectively).

[0046] The USB IN interrupt module mainly adopts a self-adaptive algorithm, Figure 5 which dynamically adjusts the calculation method of reading the data amount, and dynamically reads the appropriate amount of audio data from the ring queue, so that the data amount in the device is stable, and the data is uploaded when the system platform requests data.

[0047] Preferably, the Ping-Pong buffer procedure and the main loop decoding procedure module are carried out, which requires to start the DMA receiving interrupt mechanism in the MCU chip to complete, such as: firstly, start the DMA receiving task and the DMA receiving completion interrupt, the target buffer of the DMA transmission PDM data stream is buffer0, and the interrupt is triggered when buffer0 is full; then, in the interrupt processing function, the target buffer of the DMA receiving is first exchanged to buffer1, and then the full flag of buffer0 is set; at the same time, the full flags of buffer0 and buffer1 are determined in the main loop, and if the determination is true, the corresponding buffer array is extracted and decoded in software.

[0048] Preferably, the ring queue procedure module is implemented by the following method:

[0049] 1) The following operations are used to realize the fast writing of PCM data into the ring queue respectively:

[0050] que[wr_ptr] = wr_value

[0051] wr_ptr = (wr_ptr++) & (M-1)

[0052] wherein, when the wr_ptr increases to an integer multiple of M, wr_ptr = 0 is set, that is, the reversal of the write pointer wr_ptr is completed;

[0053] 2) The following operations are used to realize the fast reading of PCM data from the ring queue for USB uploading respectively:

[0054] rd_value = que[rd_ptr]

[0055] rd_ptr = (rd_ptr++) & (M-1)

[0056] wherein, when the rd_ptr increases to an integer multiple of M, rd_ptr = 0 is set, that is, the reversal of the read pointer rd_ptr is completed.

[0057] Preferably, for the PCM data block decoded by the main loop decoding procedure module, the direct writing mode into the ring queue is adopted in the ring queue procedure module, and in order to avoid the problem of frame loss caused by the out-of-bound of the write data pointer near the tail, the frame loss phenomenon is as shown in Figure 4 Therefore, the main method is as follows:

[0058] The sizes L of the two buffers buffer0 and buffer1 of the Ping-Pong buffer are both set to 2 k words, and the length m of each buffer after the extraction and decoding of the PCM data block is 2 k-1words (each 32-bit PDM data block is converted into 16-bit 1 frame), thus the length M of the ring queue is an integer multiple of the length m of the PCM data block, avoiding the phenomenon of losing data within the block when the ring queue is directly written with the block when the ring queue is full.

[0059] Preferably, the main loop decoding program module writes the decoded m = 2 k-1 words PCM data block into the ring queue, and updates the write pointer of the ring queue: wrptr += m.

[0060] Preferably, the USB IN interrupt program module uses the IN endpoint, and adopts an adaptive algorithm to dynamically read an appropriate amount of data from the ring queue and upload to the system in order to maintain the stability of the device data amount. The main method is as follows: first, accurately calculate the amount of data that has been collected but not uploaded; then, compare it with the amount of data in the half-full ring queue to calculate the adjustment amount; finally, dynamically increase or decrease the uploaded data amount by the adjustment amount, so that the amount of data that has been collected but not uploaded is kept in the state of about half-full ring queue. The specific method is as follows:

[0061] 1) The method for accurately calculating the amount of data that has been collected but not uploaded remain_len is as follows:

[0062] remain_len = (wr_ptr-rd_ptr) + (L-remain_DMA_len) / 2

[0063] In the formula, remain_DMA_len represents the remaining data amount that has not been received by DMA, and the unit is words.

[0064] 2) For the calculation of the adjustment amount, the difference (remain_len-M / 2) between remain_len and the half-full data amount of the ring queue is scaled by 8 times as the adjustment amount diff; that is,

[0065] diff = (remain_len-M / 2) / 8

[0066] The above method reduces data fluctuations and makes the data amount adjustment gradually synchronized.

[0067] 3) Determine the uploaded data amount sendlen by the adjustment amount. Take the uploaded data amount of 96 as the basis. If the difference is greater than 0, the uploaded data amount is sendlen = 96 + diff; if the difference is less than 0, the uploaded data amount is sendlen = 96-diff; wherein, the maximum adjustment amount needs to be limited. When the calculated adjustment amount diff is greater than 24, only take 24, that is, the range of sendlen is between 72 and 120 words.

[0068] The present application is not limited to the above-described specific embodiments, which are merely illustrative and not restrictive; those of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, which all belong to the protection of the present application.

Claims

1. A method for real-time complete data stream of a USB microphone array based on a programmable chip, characterized in that, include: The Ping-Pong Buffer module is used to provide an efficient temporary buffer to ensure the real-time reception of the microphone PDM data stream and subsequent extraction and decoding operations. Its main technical method is to open two buffer arrays of equal length, buffer0 and buffer1, and during the DMA transfer process, the two buffers are continuously and alternately received by the continuous PDM data stream output by the microphone. The main loop decoding module is used to perform software extraction and filtering on the PDM data stream output by the microphone to decode it into the required PCM data stream. This process consumes CPU resources. The main technical method is: whenever it is determined that a certain segment of the ping-pong buffer is full of PDM data, the main loop extracts and decodes the entire segment of the buffer array, and then writes the decoded PCM data block into the subsequent circular queue. The circular queue module provides sufficient memory for fast data read and write operations. Besides ensuring the real-time performance of the data stream, it also guarantees its integrity, preventing frame loss. The main method is to allocate a memory segment in the MCU with M=2 elements. N A one-dimensional array que[M] (where N is a positive integer) is used to reverse the read / write pointers (i.e., the index values ​​of array elements, referred to by wr_ptr / rd_ptr respectively) using bitwise AND operations; The USB IN interrupt module mainly uses an adaptive algorithm to dynamically read an appropriate amount of audio data from a circular queue, so that the amount of data in the device is stable and the data is uploaded when the system platform requests data.

2. The method for real-time complete data stream of a USB microphone array based on a programmable chip according to claim 1, characterized in that, The execution of the ping-pong buffer program and the main loop decoding program module requires enabling the DMA receive interrupt mechanism on the MCU chip. For example, initially, the DMA receive task and DMA receive completion interrupt are enabled. The target buffer for DMA transmission of PDM data stream is buffer0. When buffer0 is full, an interrupt is triggered. Then, in the interrupt handling function, the target buffer for DMA receive is first changed to buffer1, and then the full flag of buffer0 is set. At the same time, the main loop checks the full flags of buffer0 and buffer1. If the check is true, the corresponding buffer array is extracted and decoded in software.

3. The method for real-time complete data stream of a USB microphone array based on a programmable chip according to claim 1, characterized in that, The circular queue program module is implemented as follows: PCM data is quickly written to the circular queue using the operations que[wr_ptr] = wr_value and wr_ptr = (wr_ptr++) & (M-1). When wr_ptr increases to a multiple of M, wr_ptr is set to 0, thus reversing the write pointer wr_ptr. Similarly, PCM data is continuously and quickly read from the circular queue using the operations rd_value = que[rd_ptr] and rd_ptr = (rd_ptr++) & (M-1) for USB upload. When rd_ptr increases to a multiple of M, rd_ptr is set to 0, thus reversing the read pointer rd_ptr.

4. The method for real-time complete data stream of a USB microphone array based on a programmable chip according to claim 1, 2, or 3, characterized in that, For PCM data blocks decoded by the main loop decoding module, the circular queue module adopts a direct writing mode to the circular queue. To avoid frame drops due to write pointer out-of-bounds errors near the end of the block, the following method is used: The size L of both buffers (buffer0 and buffer1) in the ping-pong buffer is set to 2. k words, then the length m of each cached segment after decoding into a PCM data block is 2. k-1 The words (each 32-bit PDM data block is converted into a 16-bit frame) are used, so the length M of the circular queue is an integer multiple of the length m of the PCM data block, which avoids the phenomenon of data loss within the block when the data block is directly written to the circular queue and is not full.

5. The method for real-time complete data stream of a USB microphone array based on a programmable chip according to claim 1, characterized in that, The main loop decoding program module will decode m=2 k-1 After the PCM data block is written to the circular queue, update the write pointer of the circular queue: wr_ptr += m.

6. The method for real-time complete data stream of a USB microphone array based on a programmable chip according to claim 1 or 3, characterized in that, The USB IN interrupt module uses the IN endpoint. To maintain a stable amount of device data, an adaptive algorithm is employed to dynamically read an appropriate amount of data from the circular queue and upload it to the system. The main method is as follows: First, accurately calculate the amount of data that has been collected but not yet uploaded. Then, compare this amount with the amount of data in the circular queue that is half full and calculate the adjustment amount. Finally, use the adjustment amount to dynamically increase or decrease the amount of data to be uploaded, so that the amount of data that has been collected but not yet uploaded is kept at approximately half full in the circular queue. The specific steps are as follows: First, the accurate method for calculating the amount of data collected but not yet uploaded, `remain_len`, is as follows: remain_len=(wr_ptr-rd_ptr)+(L-remain_DMA_len) / 2 In the formula, remain_DMA_len represents the amount of data remaining before the DMA has finished receiving, in words; Then, for the calculation of the adjustment amount, the difference between remain_len and the half-full data volume of the circular queue (remain_len-M / 2) is scaled by 8 times as the adjustment amount diff, in order to reduce data fluctuations and make the data volume adjustment gradually return to synchronization; Finally, the upload data volume sendlen is determined by the adjustment amount. Based on the upload data volume of 96, if the difference is greater than 0, the upload data volume is sendlen = 96 + diff; if the difference is less than 0, the upload data volume is sendlen = 96 - diff. The maximum adjustment amount is limited. When the calculated adjustment amount diff is greater than 24, only 24 is taken. That is, the range of sendlen is between 72 and 120 words.

Images