A recording and playback scheduling method with a double helix structure

Abstract

A recording and playback scheduling method with a double-helix structure belongs to the field of multimedia communication technology. It includes the following steps: step 1, prepare N frame buffers for recording and playback; step 2, add N recording buffers to the sound card and start recording; write N playback buffers into the sound card one by one; step 3, write the playback buffer to the sound card in the recording callback function; step 4, add the recording buffer to the playback callback function; step 5, end the recording and playback. The present invention adopts recording callback + writing buffer to the sound card as the recording frame time consumption, and uses playback callback + adding recording buffer as the playback frame time consumption, which plays a role of neutralization and averaging, can reduce the duration jitter and dynamically tend to be consistent; it can passively synchronize the recording and playback signals under the working principle of the sound card's preemptive mode.

Description

Technical Field

[0001] The invention belongs to the technical field of multimedia communication, and in particular relates to a recording and playback scheduling method of a double-helix structure. Background Art

[0002] In the field of computer multimedia technology, the sound card is one of the most important peripherals. In addition to the board-level integrated sound card, independent sound cards are mostly integrated in the form of PCI cards. When the computer processes real-time audio, the operating system and the sound card are transferred and scheduled through DMA. That is, the sound card and DMA exchange bidirectional data through hardware interrupts, and the operating system and DMA exchange bidirectional data, which helps to free the CPU from the need to respond to the sound card hardware interrupt and improve the scheduling efficiency of the operating system.

[0003] However, this processing method has the disadvantage of increasing the delay for both recording and playback. The cache managed by DMA is called the kernel cache, and the memory used by the operating system to exchange with the kernel cache is called the application cache. The application cache cannot be too small, otherwise continuous recording and playback will not be possible, but the larger the application cache, the greater the system delay caused. On a platform with an operating system such as a computer, processing the audio reading and writing of the sound card must be at the cost of a certain delay. However, scheduling jitter will occur, and the aging of the crystal oscillator of the sound card will cause this kind of jitter noise unique to the digital system, which will have a certain impact on recording and playback. For example, if the jitter amplitude is large, the time uniformity of the audio sampling can be destroyed, the physical frequency of the sound will be distorted, and more seriously, it will cause audible interruptions in the time domain of the sound. Generally speaking, the size of the application cache can "eliminate" this jitter effect to a certain extent, especially a long enough cache.

[0004] However, in scenarios such as duplex voice communication, due to the working principle of the acoustic echo cancellation module, the echo cancellation processing of the recording signal needs to be carried out with the reference of the playback signal. At this time, how to "eliminate" the asynchrony of the two-way audio caused by system jitter is an important guarantee. In practice, when one channel is needed as a reference for another channel, a certain degree of synchronization is required. Similarly, the above problem also exists when considering multi-channel microphone signals for sound source localization. Summary of the invention

[0005] In view of the above problems existing in the prior art, the purpose of the present invention is to provide a recording and playback scheduling method with a double-helix structure, which can passively and dynamically synchronize the recording and playback signals under the working principle of the sound card's preemptive mode.

[0006] The present invention provides the following technical solution: a recording and playback scheduling method of a double-helix structure, comprising the following steps:

[0007] Step 1: Prepare N frame buffers for recording and playback respectively;

[0008] Step 2: Add N recording buffers to the sound card and start recording; write N playback buffers to the sound card one by one;

[0009] Step 3: In the recording callback function, write the playback buffer to the sound card;

[0010] Step 4: Add the recording buffer to the playback callback function;

[0011] Step 5: End recording and playback.

[0012] Furthermore, the specific instructions of step 2 are all sequential and serial.

[0013] Furthermore, in step 2, the recording buffer is empty or to be filled; the playback buffer can be filled through the network or by reading a file and then written to the sound card.

[0014] Furthermore, the specific process of step three is as follows:

[0015] In the recording callback function, write the buffer marked as i in the N playback buffer queues to the sound card. i starts from 0 and increases by 1. If i is equal to N, it is reset to 0.

[0016] Furthermore, the specific process of step 4 is as follows:

[0017] In the playback callback function, add N caches marked as j to the recording cache queue. j starts at 0 and increases by 1. If j is equal to N, it is reset to 0.

[0018] Furthermore, in step three and step four, before writing the cache marked as i in the N playback cache queues to the sound card and adding the cache marked as j in the N recording cache queues, a sub-thread is started to construct a multi-threaded recording and playback scheduling.

[0019] Furthermore, when processing the real-time playback signal, the buffer marked i in step 3 needs to be taken out for processing before being added to the playback buffer queue, and the processing result is saved;

[0020] When processing the real-time recorded signal, the buffer marked with j in step 4 should be processed before adding it to the sound card.

[0021] Furthermore, in step three and step four, the recording callback function and the playback callback function respectively execute writing the playback buffer to the sound card or adding the recording buffer; and after adding the recording buffer and writing the buffer to the sound card, the corresponding recording callback function and the playback callback function can be called; the calling of the recording callback function and the playback callback function, writing the playback buffer to the sound card in the recording callback function, and adding the recording buffer to the playback callback function are repeated until the recording and playback are completed.

[0022] By adopting the above technology, compared with the prior art, the beneficial effects of the present invention are as follows:

[0023] The present invention adopts recording callback + writing to the sound card cache as the recording frame time consumption, and uses playback callback + adding to the recording cache as the playback frame time consumption, which plays a role of neutralization and averaging, can reduce the duration jitter and dynamically tend to be consistent; and can passively dynamically synchronize the recording and playback signals under the working principle of the sound card's preemptive mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A system block diagram of the overall method of the present invention;

[0025] Figure 2 The schematic diagram of the principle of repeatedly looping between calling the recording callback function and the playback callback function, writing the playback buffer to the sound card in the recording callback function, and adding the recording buffer to the playback callback function in step 3 and step 4 of the present invention;

[0026] Figure 3 This is a schematic diagram of the analogy between the recording and playback timeline of the present invention and the DNA structure. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments of the specification. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0028] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0029] See also Figure 1-3 :

[0030] Embodiment 1

[0031] A double-helix structure recording and playback scheduling method, comprising the following steps:

[0032] Step 1: Prepare N frame buffers for recording and playback respectively;

[0033] Step 2, add N recording buffers to the sound card and start recording; write N playback buffers into the sound card one by one; wherein the specific sub-steps of step 2 are all sequential serial; the recording buffer is empty or to be filled; the playback buffer can be filled through the network or by reading a file and then written into the sound card.

[0034] Step 3: In the recording callback function, write the buffers marked as i in the N playback buffer queues to the sound card. i starts from 0 and increases by 1. If i is equal to N, it is reset to 0.

[0035] Step 4: In the playback callback function, add N recording cache queues with a cache marked as j. j starts at 0 and increases by 1. If j is equal to N, it is reset to 0.

[0036] Step 5: End recording and playback

[0037] Among them, in step three and step four, if the signal processing module is required to process the real-time recording and playback signal, such as filtering before playback, noise reduction after recording, or echo elimination, then just take out the cache marked i in step three for processing before adding it to the recording and playback cache queue, and save the processing result, which is the pre-playback processing; or process the cache marked j in step four before or after writing and adding it to the sound card, which is the post-recording filtering.

[0038] In step three and step four, the recording callback function and the playback callback function respectively execute writing the playback buffer to the sound card or adding the recording buffer; after adding the recording buffer and writing the buffer to the sound card, the corresponding recording callback function and the playback callback function can be called; the calling of the recording callback function and the playback callback function, writing the playback buffer to the sound card in the recording callback function, and adding the recording buffer to the playback callback function are repeated until the recording and playback are completed.

[0039] Embodiment 2

[0040] The difference between this embodiment and the first embodiment is that:

[0041] In step 3 and step 4, before writing the buffer marked as i in the N playback buffer queues to the sound card and before adding the buffer marked as j in the N recording buffer queues, a child thread is started to build a multi-threaded recording and playback scheduling.

[0042] Because the two callback functions are blocking and cannot be executed for too long, it is convenient to insert the signal processing filter module when using multi-threaded mode to perform the operations of "adding recording buffer" and "writing buffer to sound card".

[0043] The following table is an example of the running results; the program design specification of the recording and playback frame length is 20 milliseconds (ms):

[0044]

[0045] From the five experiments in the table above, we can see that the average frame length is not exactly 20 milliseconds due to the existence of scheduling jitter. Secondly, the synchronization of recording and playback is improved compared with the normal mode. The consistency of the number of frames of recording and playback is much better than that of the normal mode (the total number of frames differs by 14 frames). It can be seen that the two are synchronized. This consistency of the number of frames will affect the performance of the signal processing and filtering modules, especially modules such as acoustic echo cancellation. The update frequency of the echo filter is neither reduced nor increased, because the number of recording frames basically matches the number of playback frames, thus ensuring the stability of the filtering effect.

[0046] In the present invention, "adding recording cache", "writing cache to sound card", "recording callback" and "playback callback" are equivalent to the four pairs of bases in ATCG, and their sequence progressive combination evolution is completely consistent with the function calling pattern on the two timelines of recording and playback, so this method is called "double helix structure recording and playback scheduling".

[0047] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A double-helix structure recording and playback scheduling method, characterized in that: The following steps are involved: Step 1: Prepare N frame buffers for recording and playback respectively; Step 2: Add N recording buffers to the sound card and start recording; write N playback buffers to the sound card one by one; Step 3: In the recording callback function, write the playback buffer to the sound card; the specific process of step 3 is as follows: In the recording callback function, write the buffer marked as i in the N playback buffer queues to the sound card. i starts from 0 and increases by 1. If i is equal to N, it is reset to 0. Step 4: Add the recording buffer to the playback callback function. The specific process of step 4 is as follows: In the playback callback function, add N recording cache queues with the cache marked as j. j starts at 0 and increases by 1. If j is equal to N, it is reset to 0. Step 5: End recording and playback; In step 3 and step 4, the recording callback function and the playback callback function respectively execute writing the playback buffer to the sound card or adding the recording buffer; and after adding the recording buffer and writing the buffer to the sound card, the corresponding recording callback function and the playback callback function can be called; the calling of the recording callback function and the playback callback function, writing the playback buffer to the sound card in the recording callback function, and adding the recording buffer in the playback callback function are repeated until the recording and playback are completed.

2. A double-helix structure recording and playback scheduling method according to claim 1, characterized in that: The specific instructions of step 2 are all sequential and serial.

3. A double-helix structure recording and playback scheduling method according to claim 2, characterized in that: In the step 2, the recording buffer is empty or to be filled; the playback buffer can be filled through the network or by reading a file and then written into the sound card.

4. A double-helix structure recording and playback scheduling method according to claim 1, characterized in that: In step 3 and step 4, before writing the cache marked as i in the N playback cache queues to the sound card and adding the cache marked as j in the N recording cache queues, a sub-thread is started to build a multi-threaded recording and playback scheduling.

Images