Multi-physical sound card audio data playback method, system, device and product

Abstract

The present invention discloses a method, system, device and product for playing audio data of multiple physical sound cards. The method comprises: the playing thread writes the audio data to be played into the cache of the virtual sound card layer; the multimedia digital signal codec in each physical sound card obtains the audio data to be played in the cache according to the clock cycle corresponding to the working clock and plays it, wherein the working clock is obtained by dividing the main clock provided by the same crystal oscillator to each physical sound card; based on the working clock and the preset cache capacity interval of the virtual sound card layer, the amount of audio data written into the cache by the playing thread and the amount of audio data read from the cache based on the clock cycle are controlled so that the error does not exceed a preset threshold. Through the technical solution of the present invention, the problems of asynchronous playback of the same-source sound by multiple sound card devices, too long playback delay, and noise in some sound card playback devices due to rhythm problems during playback can be solved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of audio technology, and in particular to a method, system, device and product for playing audio data of multiple physical sound cards. Background Art

[0002] When multiple physical sound card devices play the same sound source, multiple sound card drivers work independently, and the application performs playback operations in multi-threaded or single-threaded mode. When working in multi-threaded mode, a single thread independently controls a single sound card. Due to different thread scheduling and different working states of sound card drivers, it is impossible to synchronize multiple sound cards when playing the same sound source, resulting in different sounds on multiple playback devices. When single-threaded control is used, since all sound card data is written by the same thread, data synchronization can be achieved during input. However, since each sound card works independently, the working rhythm of the playback thread will be affected. Some sound cards will even block the playback thread for a long time, resulting in different times for the thread to write the same data to each sound card. In the end, it is still impossible to synchronize the playback of multiple sound card devices. When multiple sound cards play the same sound source, it is particularly prominent in the case of live sound reinforcement. The experience caused by the lack of synchronization will be very poor, and noise will appear if it is not well controlled. Summary of the invention

[0003] The embodiments of the present invention provide a method, system, device and product for playing audio data of multiple physical sound cards, so as to solve the problems of asynchronism, long delay and noise in some sound card playback devices caused by rhythm problems when multiple sound card devices play the same sound.

[0004] According to one aspect of the present invention, a method for playing audio data of multiple physical sound cards is provided, comprising:

[0005] The playback thread writes the audio data to be played into the cache of the virtual sound card layer;

[0006] The multimedia digital signal codec in each of the physical sound cards obtains the audio data to be played in the cache and plays it according to the clock cycle corresponding to the working clock, wherein the working clock is obtained by dividing the main clock provided by the same crystal oscillator to each of the physical sound cards;

[0007] Based on the working clock and the preset cache capacity interval of the virtual sound card layer, the error between the amount of audio data written into the cache by the playback thread and the amount of audio data read from the cache based on the clock cycle does not exceed a preset threshold.

[0008] According to another aspect of the present invention, a multi-physical sound card audio data playback system is provided, the system comprising: a virtual sound card layer, a crystal oscillator and at least two physical sound cards; the physical sound card comprises: a multimedia digital signal codec; each of the multimedia digital signal codecs is connected to the crystal oscillator;

[0009] The virtual sound card layer is used to cache the audio data to be played written by the playback thread within a preset cache capacity interval;

[0010] The multimedia digital signal codec is used to obtain the audio data to be played in the buffer of the virtual sound card layer and play it according to the clock cycle corresponding to the working clock, wherein the working clock is obtained by dividing the main clock provided by the crystal oscillator to each of the physical sound cards;

[0011] The error between the amount of audio data written into the cache by the playback thread and the amount of audio data read from the cache based on the clock cycle is controlled to not exceed a preset threshold.

[0012] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the multi-physical sound card audio data playing method described in any embodiment of the present invention.

[0016] According to another aspect of the present invention, an embodiment of the present invention further provides a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, it implements the multi-physical sound card audio data playback method described in any embodiment of the present invention.

[0017] In the embodiment of the present invention, the audio data to be played is written into the cache of the virtual sound card layer by the playing thread, and the multimedia digital signal codec in each physical sound card obtains the audio data to be played in the cache according to the clock cycle corresponding to the working clock and plays it. Since the working clock is obtained by dividing the main clock provided by the same crystal oscillator to each physical sound card, each physical sound card can keep synchronization when obtaining the audio data to be played from the virtual sound card layer, and then the error between the amount of audio data written into the cache by the playing thread and the amount of audio data read from the cache based on the clock cycle does not exceed the preset threshold value based on the working clock and the preset cache capacity interval of the virtual sound card layer, so that the amount of data written into the cache is equal to the amount of data read from the cache, so that a certain amount of audio data to be played can be stored in the cache of the virtual sound card layer for the physical sound card to read, and there will be no delay caused by too much cache data or noise caused by too little cache data. Through the technical solution of the present invention, the problem of asynchronism when multiple sound card devices play the same source sound, the problem of too long delay, and the problem of noise in some sound card playback devices caused by rhythm problems can be solved.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 is a flow chart of a method for playing audio data of multiple physical sound cards in an embodiment of the present invention;

[0021] Figure 2 It is a structural schematic diagram of a multi-physical sound card audio data playback system in an embodiment of the present invention;

[0022] Figure 3 is a schematic structural diagram of an exemplary multi-physical sound card audio data playback system in an embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of a data flow process in which a playback thread participates in an embodiment of the present invention;

[0024] Figure 5is a schematic diagram of a data flow process in which a collection thread participates in an embodiment of the present invention;

[0025] Figure 6 It is a structural schematic diagram of an electronic device for implementing the multi-physical sound card audio data playing method of an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0029] Embodiment 1

[0030] Based on the prior art, it is known that the reasons why multiple physical sound cards cannot be synchronized when playing the same sound source can mainly include the following reasons: First, the CPU scheduling situation. Because of multi-threading, running on the CPU, affected by CPU scheduling, the rhythm of multiple threads running cannot be consistent. Second, the rhythm of hardware playback. This rhythm depends on the hardware design of the physical sound card. In general designs, the designs of each physical sound card are independent, and there is no correlation in the clock, so the rhythm of playback is also not related.

[0031] In summary, the core problem that needs to be solved is: the clocks on the hardware have deviations, and there are deviations between them. In addition, due to the CPU scheduling delay, a lot of time is uncontrollable, so synchronization based on pure software is unscientific. Therefore, it is necessary to first improve the hardware, and then achieve synchronization when multiple physical sound cards play the same sound source based on software and hardware improvements.

[0032] Figure 1 1 is a flowchart of a method for playing audio data of multiple physical sound cards in an embodiment of the present invention. This embodiment is applicable to the case where multiple physical sound card devices play audio data of the same sound source synchronously. The method can be executed by a system for playing audio data of multiple physical sound cards in an embodiment of the present invention. The system can be implemented in software and / or hardware, such as Figure 1 As shown, the method specifically comprises the following steps:

[0033] S101, the playing thread writes the audio data to be played into the cache of the virtual sound card layer.

[0034] In this embodiment, a pure software virtual sound card layer is added to the Linux kernel to facilitate the design of application-state programs. It can take effect on specific physical sound cards with unified parameters, abstract multiple physical sound cards into different channels of the virtual sound card layer, and facilitate the synchronous control of multiple physical sound cards playing the same sound source sound.

[0035] In this embodiment, a buffer pool (hereinafter referred to as a buffer) is provided in the virtual sound card layer for storing audio data to be played, wherein the audio data to be played is the audio data output after the audio data decoded by the network and collected locally is input into the audio mixer.

[0036] In this embodiment, the play thread is a program that writes the audio data to be played into the buffer of the virtual sound card layer.

[0037] S102: The multimedia digital signal codec in each physical sound card obtains the audio data to be played in the buffer according to the clock cycle corresponding to the working clock and plays the audio data.

[0038] Among them, the physical sound card is generally composed of two parts: the physical part and the physical sound card driver part at the software level. The physical part usually refers to the hardware device used to process audio signals in a computer or audio device. It is responsible for converting analog audio signals into digital signals, and converting digital audio signals into analog signals for playback in speakers or headphones. In one embodiment, the physical part of the physical sound card specifically includes: codec (abbreviation of Coder / DECoder, multimedia digital signal codec), I2C (Inter-Integrated Circuit, integrated circuit bus), I2S bus (Inter-IC Sound, integrated circuit built-in audio bus) and the I2S controller inside the CPU side.

[0039] In this embodiment, the multimedia digital signal codec is also called codec, and the codec is used to realize the conversion between digital signals and analog signals, including converting digital signals into analog signals and converting analog signals into digital signals.

[0040] In this embodiment, the working clock refers to the clock used when the codec transmits audio data to the virtual sound card layer, and the working clock is obtained by dividing the master clock provided by the crystal oscillator to the physical sound card. Specifically, each physical sound card is connected to the same crystal oscillator. This embodiment does not limit the model and other information of the crystal oscillator. The same crystal oscillator provides a master clock to each physical sound card, and the master clock provided by the crystal oscillator is divided to obtain the working clock of the codec in each physical sound card. In one embodiment, the crystal oscillator provides a 12.288M master clock to the codec in the physical sound card, and the codec will divide the 12.288M clock to obtain the working clock of the physical sound card when it is working. For example, the codec divides a 48k working clock from the 12.288M clock.

[0041] Since the working clock of each physical sound card is obtained by dividing the master clock provided by the same crystal oscillator, each working clock shares a master clock source, and they all maintain the same phase and frequency as the master clock, therefore, the working clock of each physical sound card is synchronized. It should be noted that, in this embodiment, the working clock of the physical sound card is used as the master clock for controlling the audio data transmission process, that is, the working rhythm of each physical sound card carrying data from the virtual sound card layer and the working rhythm of writing data in each physical sound card into the virtual sound card layer are affected by the working clock of the corresponding physical sound card, and since the working clock of each physical sound card is synchronized, it is ensured that the working rhythm of each physical sound card carrying data from the virtual sound card layer and the working rhythm of writing data in each physical sound card into the virtual sound card layer are synchronized.

[0042] In this embodiment, the clock cycle refers to the clock cycle corresponding to the working clock of the codec.

[0043] Specifically, because the codec plays audio data at the frequency of the working clock, and because the working clock of the codec is the main clock that controls the audio data transmission process, the physical sound card driver is forced to read the audio data to be played from the cache according to the clock cycle of the working clock and transmit it to the codec through the I2S bus, ensuring that the codec can obtain the audio data to be played in the cache according to the clock cycle of the working clock for playback. Since the working clock of the codec of each physical sound card is synchronized, playback synchronization can be achieved.

[0044] S103, based on the working clock and the preset buffer capacity interval of the virtual sound card layer, control the difference between the amount of audio data written into the buffer by the playback thread and the amount of audio data read from the buffer based on the clock cycle to not exceed a preset threshold.

[0045] The cache capacity interval refers to the capacity range of the audio data volume of the audio data to be played that can be stored in the cache of the virtual sound card layer. The cache capacity interval controls the amount of audio data to be played written by the playback thread into the cache of the virtual sound card layer. The cache capacity interval consists of two boundary values, which are obtained based on calculations. The amount of audio data can be calculated based on the number of audio frames, the frame duration of each frame of audio data, the sampling rate, and the sampling bit width, and is used to represent the amount of data contained in the audio data.

[0046] It should be noted that, generally speaking, the running rhythm of the playback thread is affected by the rhythm of the audio mixer and the CPU scheduling. Furthermore, in this embodiment, based on the above two influencing factors, two control factors are added to control the running rhythm of the playback thread, one is the running rhythm of the working clock, which is determined by the data statistics of the acquisition thread; the other is the maximum value in the buffer capacity interval of the virtual sound card layer during playback.

[0047] In this embodiment, the physical sound card will continuously read audio data after it is started, and the working rhythm of the playback thread writing the cache is controlled by the cache capacity interval and the working clock. The rhythm of the working clock controls the rhythm of the playback thread writing the audio data into the cache, and the cache capacity interval controls the amount of audio data written into the cache each time; thereby controlling the error between the cumulative amount of audio data written into the cache by the playback thread and the cumulative amount of audio data read from the cache by the physical sound card not to exceed a preset threshold, where the error can be understood as the difference between the amounts of audio data, thereby ensuring that the cache of the virtual sound card layer always stores audio data to be played for the physical sound card to read, and at the same time, there will be no delay caused by too much audio data stored in the cache, and no noise caused by too little audio data stored in the cache.

[0048] In this embodiment, the preset threshold value can be set according to actual needs or experience values, and this embodiment does not specifically limit the preset threshold value. In actual operation, the preset threshold value can be set as small as possible. The smaller the preset threshold value is, the closer the amount of audio data written into the cache by the playback thread is to the amount of audio data read from the cache.

[0049] In the embodiment of the present invention, the audio data to be played is written into the cache of the virtual sound card layer by the playing thread, the main clock provided by the same crystal oscillator to each physical sound card is divided to obtain the working clock, and the multimedia digital signal codec in each physical sound card obtains the audio data to be played in the cache and plays it according to the clock cycle corresponding to the working clock, so that each physical sound card can keep synchronization when reading the audio data to be played from the virtual sound card layer, and then the error between the amount of audio data written into the cache by the playing thread and the amount of audio data read from the cache based on the clock cycle does not exceed the preset threshold value based on the working clock and the preset cache capacity interval of the virtual sound card layer, so that the amount of data written and the amount of data read from the cache tend to be equal, so that a certain amount of audio data to be played can be stored in the cache of the virtual sound card layer for the physical sound card to read, and at the same time, there will be no delay caused by too much cache data or noise caused by too little cache data. Through the technical solution of the present invention, the problem of asynchronism when multiple sound card devices play the same source sound, the problem of too long delay, and the problem of noise in some sound card playback devices caused by rhythm problems can be solved.

[0050] Optionally, in one embodiment, controlling the difference between the amount of audio data written into the cache by the playback thread and the amount of audio data read from the cache based on the clock cycle based on the working clock and the preset cache capacity interval of the virtual sound card layer does not exceed a preset threshold, including:

[0051] (1) Based on the clock cycle of the working clock, the target data in the physical sound card is moved to the virtual sound card layer.

[0052] It should be explained that the target data refers to the data collected by the physical sound card. Exemplarily, the target data may be audio data or other data, which is not limited in this embodiment.

[0053] Specifically, the multimedia digital signal codec in each physical sound card moves the target data in the physical sound card to the virtual sound card layer according to the clock cycle corresponding to the working clock.

[0054] (2) The acquisition thread reads the target data from the virtual sound card layer and determines the first frame number of the target data that the acquisition thread has read cumulatively up to the current statistical time.

[0055] In this embodiment, the acquisition thread is a program that reads target data from the cache of the virtual sound card layer, and the working rhythm of the acquisition thread is controlled by the working clock of the multimedia digital signal codec in the physical sound card. Specifically, when the acquisition thread of the application reads the target data of the virtual sound card layer, if there is data written into the cache of the virtual sound card layer based on the working clock, the data will be returned. If there is no data written into the cache of the virtual sound card layer based on the working clock, the acquisition thread will be blocked (sleep), so the rhythm of the acquisition thread is controlled by the working clock, that is, the acquisition rhythm of the acquisition thread is the acquisition or playback rhythm of the multimedia digital signal codec (codec), so theoretically, the cumulative number of frames that the acquisition thread can collect so far is the same as the cumulative number of frames that the codec can read from the cache of the virtual sound card layer at the same time, so the number of frames of the audio data to be played written into the cache of the virtual sound card layer by the playback thread can be controlled based on the number of frames collected by the acquisition thread, so that the codec can read the required audio data for playback when running according to the working clock.

[0056] The number of frames obtained after the acquisition thread starts running is a cumulative value. Therefore, the number of the first frame obtained here is the number of frames of target data that the acquisition thread has accumulated from the virtual sound card layer up to the current statistical time.

[0057] Specifically, when the acquisition thread counts the number of acquired frames, it will not simultaneously execute the step of acquiring the number of frames from the cache. Therefore, the acquisition thread can count the number of frames that have actually been acquired up to the current statistical time, that is, it can obtain the first frame number corresponding to the target data accumulated and read from the virtual sound card layer up to the current statistical time recorded by the acquisition thread during the acquisition process.

[0058] (3) When the playback thread starts the current running round, the number of second frames of the audio data to be played that have been written into the cache by the playback thread up to the current statistical time is obtained.

[0059] Similar to the acquisition thread, the number of frames obtained after the playback thread starts running is also a cumulative value. Therefore, the second frame number obtained here is the number of frames of audio data to be played that the playback thread has accumulated and written into the cache as of the current statistical time.

[0060] In this embodiment, there is only one playback thread, which runs multiple times, and writes a certain number of frames of audio data into the cache each time it runs. Specifically, the logic for determining the number of frames of audio data written each time is as follows: when the current running round starts, the playback thread obtains the cumulative value of the audio frames corresponding to the audio data to be played that the playback thread has accumulated and written into the cache as of the current statistical time, that is, the second frame number.

[0061] (4) Determine the initial number of difference frames based on the start time of the acquisition thread and the playback thread.

[0062] The acquisition thread and the playback thread may not be started at the same time, so when the latter is started, there will be a frame number difference between the two threads, and this frame number difference is the initial difference frame number. For example, the acquisition thread and the playback thread are not started at the same time, and the playback thread is started first to write data to the cache. When the acquisition thread is started later, the number of frames corresponding to the audio data written into the cache by the playback thread at this time is the initial difference frame number.

[0063] Specifically, the initial difference frame number can be determined by calculating the number of frames of audio data written into the cache by the playback thread in the time period between the two startup times according to the startup time of the acquisition thread and the playback thread.

[0064] (5) When the third frame number obtained by subtracting the initial difference frame number from the second frame number is less than the first frame number, the target frame number to be written into the cache in the current running round of the playback thread is determined based on the first frame number, the third frame number, and the cache capacity interval.

[0065] It should be noted that the target number of frames is the number of frames of audio data to be played written to the virtual sound card layer as of the current statistical time, after removing the initial difference frame number. When the number of frames of audio data to be played written to the virtual sound card layer is less than the number of frames of target data read from the virtual sound card layer, the playback thread should write the number of frames of audio data to be played in the cache of the virtual sound card layer in the current running round.

[0066] Specifically, as of the current statistical time, when the number of the third frame is less than the number of the first frame, it means that the number of audio frames written by the playback thread to the virtual sound card layer cache is less than the number of audio frames read by the physical sound card from the virtual sound card layer cache, thereby determining that the rhythm of the playback thread writing the audio data to be played into the cache is slower than the rhythm of the hardware sound card playing the audio data, which may easily cause no data consumption in the virtual sound card layer cache, resulting in noise. At this time, the target number of frames should be determined based on the difference between the playback thread and the acquisition thread and the cache capacity interval, and the number of frames of the data to be played written into the cache in the current running round of the playback thread should be determined as the target number of frames, narrowing the gap between the number of audio frames written into the cache by the playback thread and the number of audio frames collected by the acquisition thread, and based on the cache capacity interval, controlling the amount of audio data in the cache after the playback thread writes the target number of frames into the cache to fall within the cache capacity interval to avoid delays.

[0067] The technical solution of the embodiment of the present invention compares the number of first frames of target data read cumulatively from the virtual sound card layer by the acquisition thread as of the current statistical time with the difference between the number of second frames of audio data to be played written cumulatively into the cache by the playback thread as of the current statistical time when the playback thread starts the current running round minus the initial difference frame number, and controls the running rhythm of the playback thread based on the comparison result and the cache capacity interval, thereby preventing the playback thread from running too slowly and avoiding delays.

[0068] Optionally, in one implementation, the virtual sound card layer is pre-constructed based on the number of physical sound cards, the number of channels of each physical sound card, and a preset arrangement of each physical sound card.

[0069] Generally, a physical sound card has several channels, and the number of channels of each physical sound card can be the same or different. For example, physical sound card 1 has 4 channels, physical sound card 2 has 2 channels, and physical sound card 3 has 2 channels. In this embodiment, the arrangement of each physical sound card can be set by the user according to the needs. For example, the arrangement can be physical sound card 1, physical sound card 2, and physical sound card 3. In this embodiment, multiple physical sound cards are abstracted into different channels of a virtual sound card layer. The number of channels of the virtual sound card layer is equal to the sum of the channels of all physical sound cards, and the arrangement of the channels in the virtual sound card layer corresponds to the arrangement of the physical sound cards.

[0070] Before the playback thread writes the audio data to be played into the cache of the virtual sound card layer, it also includes:

[0071] Configure the property information of the virtual sound card layer and synchronize the property information of the virtual sound card layer to each physical sound card.

[0072] The attribute information includes sampling rate, sampling bit width and cache capacity range.

[0073] Specifically, in this embodiment, after the virtual sound card layer is constructed, the user can configure the attribute information of the virtual sound card layer through the application, and the attribute information includes sampling rate, bit width, and cache capacity interval. The virtual sound card layer processes the configured information, including generating address information of the cache capacity, which is needed when the physical sound card stores data in the cache of the virtual sound card layer or reads data from it; the virtual sound card layer synchronously configures the sampling rate, sampling bit width, cache capacity address information, and cache capacity interval to each specific physical sound card to ensure that the parameters of each physical sound card are consistent.

[0074] The technical solution of the embodiment of the present invention constructs a virtual sound card layer according to each physical sound card, abstracts the operations in each physical sound card into a unified operation object, abstracts each physical sound card into different channels of the virtual sound card layer, and configures the attribute information of the configured virtual sound card layer into each physical sound card, so as to ensure that the parameters of each physical sound card are consistent. The user can uniformly control each physical sound card through the abstract layer of the virtual sound card layer in the application, and no longer needs to pay attention to each specific physical sound card. It only needs to operate the abstract virtual sound card layer, which saves the configuration of the parameters of each physical sound card and avoids the occurrence of configuration errors when configuring the parameters of each physical sound card.

[0075] Optionally, in one embodiment, determining the target number of frames to be written into the cache in the current running round of the playback thread based on the first number of frames, the third number of frames, and the cache capacity interval includes:

[0076] (1) Subtract the number of the first frame from the number of the third frame to obtain the number of difference frames.

[0077] It should be noted that the number of difference frames refers to the difference between the cumulative number of frames read by the acquisition thread and the cumulative number of frames written by the playback thread as of the current statistical time, after removing the influence of the initial number of difference frames.

[0078] Specifically, when comparing the number of frames corresponding to the acquisition thread and the number of frames corresponding to the playback thread as of the current statistical time after removing the influence of the initial difference frame number, the comparison can be based on the number of frames of a single channel.

[0079] (2) Determine the capacity corresponding to the number of difference frames according to the number of difference frames, the preset frame duration, the sampling rate, the sampling bit width, and the number of channels of the virtual sound card layer.

[0080] The frame duration is used to indicate the frame size of the audio data, which is determined by a time period. For example, a frame duration of 5 ms means 5 ms of audio data. The specific frame duration can be set by the user according to actual conditions or experience, and this embodiment does not limit this.

[0081] It should be noted that the capacity corresponding to the number of difference frames refers to the cache capacity required to store the amount of audio data corresponding to the audio frames of the number of difference frames.

[0082] Specifically, the amount of audio data corresponding to the number of difference frames may be determined by multiplying the number of difference frames, the preset frame duration, the sampling rate, the sampling bit width, and the number of channels of the virtual sound card layer, thereby determining the required cache capacity.

[0083] (3) Obtain the remaining cache capacity in the cache based on the cache capacity interval.

[0084] The remaining buffer capacity may be the remaining buffer capacity in the buffer of the virtual sound card layer.

[0085] Specifically, the maximum value in the cache capacity interval is obtained, the amount of audio data written in the cache is obtained, and the two are subtracted to obtain the remaining cache capacity in the cache.

[0086] (4) If the remaining cache capacity in the cache is greater than or equal to the capacity corresponding to the number of difference frames, the target number of frames written into the cache in the current running round of the playback thread is determined to be the number of difference frames.

[0087] Specifically, if the remaining cache capacity in the cache is greater than or equal to the capacity corresponding to the number of difference frames, that is, the remaining cache capacity in the cache of the virtual sound card layer can directly put the audio data to be played with the number of difference frames, then it is determined that the current running round of the playback thread directly writes the audio data to be played with the number of difference frames into the cache of the virtual sound card layer.

[0088] (5) If the remaining cache capacity in the cache is less than the capacity corresponding to the number of difference frames, the target number of frames written into the cache in the current running round of the playback thread is determined to be the number of frames corresponding to the remaining cache capacity in the cache.

[0089] The number of frames corresponding to the remaining buffer capacity in the buffer is determined by the remaining buffer capacity in the buffer, the frame duration, the sampling rate, the sampling bit width, and the number of channels of the virtual sound card layer. It can be understood that the process of calculating the number of frames according to the capacity is the reverse process of calculating the capacity according to the number of frames, and the calculation process is not repeated in this embodiment.

[0090] Specifically, if the remaining cache capacity in the cache is less than the capacity corresponding to the number of difference frames, that is, the remaining cache capacity in the cache of the virtual sound card layer cannot hold the number of difference frames of audio data to be played at one time, then the playback thread can first write the number of frames of audio data to be played corresponding to the remaining cache capacity in the cache into the cache of the virtual sound card layer in the current running round. As the physical sound card consumes the audio data to be played in the cache of the virtual sound card layer, when remaining cache space appears in the cache of the virtual sound card layer, when the playback thread enters the next running round, the above process is repeated to determine the target number of frames that can be written into the cache in the current running round.

[0091] The technical solution of the embodiment of the present invention is that as of the current statistical time, when the number of third frames is less than the number of first frames, the capacity required for the difference frame number between the first frame number and the third frame number is compared with the remaining cache capacity of the virtual sound card layer to determine the target frame number of the audio data to be played that should be written into the cache in the current running round of the playback thread, thereby adjusting the rhythm of the playback thread to prevent the playback thread from running too slowly, while controlling the amount of audio data in the cache not to exceed the maximum value and cause delays, and jointly controlling the rhythm of the playback thread by collecting the maximum value of the thread and the cache capacity interval, so that the amount of data written into the cache by the playback thread falls exactly within the cache capacity interval of the virtual sound card layer.

[0092] Optionally, in one embodiment, controlling the difference between the amount of audio data written into the cache by the playback thread and the amount of audio data read from the cache based on the clock cycle based on the working clock and the preset cache capacity interval of the virtual sound card layer does not exceed a preset threshold, including:

[0093] When the amount of audio data written into the cache in the current running round of the playback thread is greater than the maximum value of the cache capacity interval, the playback thread is controlled to enter a blocked state and stop writing the audio data to be played into the cache until the amount of audio data in the cache is less than the maximum value of the cache capacity interval, at which time the playback thread is woken up to enter the next running round.

[0094] In this embodiment, the blocking state may be a state in which the playback thread stops writing the audio data to be played into the cache of the virtual sound card layer, which may also be called a sleeping state.

[0095] The maximum value of the cache capacity range refers to the maximum amount of audio data that can be accommodated in the cache of the virtual sound card layer, which is determined based on the sampling rate and sampling bit width set by the user, the total audio processing delay required by the product, and the number of channels of the virtual sound card layer. The maximum value limits the maximum amount of audio data to be played that can be accommodated in the cache of the virtual sound card layer, so as to avoid the delay caused by the physical sound card not reading in time due to too much audio data in the cache. At the same time, it also limits the rhythm of the playback thread writing to the cache to be faster than the actual reading rhythm of the hardware (the acquisition rhythm of the acquisition thread).

[0096] Specifically, when the amount of audio data written into the cache in the current running round of the playback thread is greater than the maximum value of the cache capacity interval, it indicates that the audio data to be played can no longer be written into the cache of the virtual sound card layer, indicating that the playback rhythm of the playback thread has exceeded the actual playback rhythm of the hardware. At this time, the playback thread can be controlled to enter a blocked state and stop writing the audio data to be played into the cache of the virtual sound card layer. As the physical sound card consumes the audio data to be played in the cache of the virtual sound card layer, until the amount of audio data in the cache is less than the maximum value of the cache capacity interval, the playback thread is awakened to enter the next running round and continue to write the audio data to be played into the cache of the virtual sound card layer.

[0097] The technical solution of the embodiment of the present invention controls the playback thread to enter a blocked state and stops writing the audio data to be played into the cache when the amount of audio data written into the cache in the current running round of the playback thread is greater than the maximum value of the cache capacity interval. This can control the speed at which the playback thread writes into the cache not too fast, so that the amount of audio data in the cache of the virtual sound card layer is not too much, thereby avoiding the problem of delay caused by the excessive amount of audio data in the cache of the virtual sound card layer resulting in untimely reading of the physical sound card.

[0098] Optionally, the cache capacity interval is determined by a minimum value and a maximum value.

[0099] In this embodiment, the minimum value can be represented by start, the maximum value can be represented by end, and the cache capacity interval can be represented as [start, end].

[0100] Among them, the minimum value is the amount of audio data read by the physical sound card during two adjacent scheduling processes of the playback thread when the CPU (i.e., central processing unit) is under high load. That is, it can be understood that the minimum value can be the amount of audio data read by the physical sound card during the period when the playback thread is scheduled by other applications to run again to write data to the cache of the virtual sound card layer when the CPU (i.e., central processing unit) is under high load. In other words, it ensures that during the period when the playback thread is called by other programs but does not write audio data to the cache of the virtual sound card layer, the data in the cache can support the demand for the physical sound card to read audio data, avoiding the occurrence of noise when the physical sound card reads audio data due to the absence of audio data in the cache.

[0101] The minimum value is determined by: obtaining the scheduling delay corresponding to two adjacent scheduling processes of the playback thread when the central processor is under high load; and determining the minimum value according to the sampling rate, scheduling delay, sampling bit width and the number of channels of the virtual sound card layer.

[0102] Among them, when the CPU is under high load, the corresponding scheduling delay of the playback thread in two adjacent scheduling processes can be obtained through actual monitoring.

[0103] In this embodiment, the start value is a value indicating the size of the cache capacity, and the calculation process can be expressed as follows: (sampling rate / 1000) × scheduling delay time (unit: ms) × sampling bit width × number of channels of the virtual sound card layer. For example, if the audio playback parameters are configured as 48k, 16bit, the number of channels of the virtual sound card layer is 16, and the scheduling delay time when the CPU is highly loaded is 5ms, then the calculation method of the start value can be expressed as start=(48000 / 1000)×5×(16 / 8)×16, and the final calculated start value is 7680byte.

[0104] The start value is set to prevent the problem of intermittent lack of audio data and noise caused by the failure to write audio data into the cache of the virtual sound card layer in time during the period when the playback thread is called away when the CPU is under high load, resulting in no audio data being played on the physical sound card.

[0105] In this embodiment, the maximum value is determined by: obtaining the audio processing delay; and determining the maximum value according to the sampling rate, the audio processing delay, the sampling bit width, and the number of channels of the virtual sound card layer.

[0106] The audio processing delay is the delay time obtained by subtracting the existing delay at the application level from the total delay requirement of the final product playback; then the maximum value is calculated based on the audio processing delay, sampling rate, sampling bit width and number of channels. For example, if the total delay requirement of the final product playback is 80ms, and the existing delay in the application processing flow buf is 50ms, then the audio processing delay in the virtual sound card layer is 30ms. Similar to the calculation of the start value, (sampling rate / 1000) × audio processing delay (in ms) × sampling bit width × number of channels is calculated to finally get the maximum value.

[0107] In one embodiment, after the playing thread writes the to-be-played audio data into the cache of the virtual sound card layer, the method further includes:

[0108] When the amount of audio data in the buffer of the virtual sound card layer is greater than the minimum value, each physical sound card is started at the same time.

[0109] Specifically, the user first configures the start value and end value through the application, and then the playback thread starts to write the audio data into the cache of the virtual sound card layer. When the amount of audio data in the cache reaches the start value, the virtual sound card layer starts to start the physical sound card and starts to transmit data outward. The transmission speed of each physical sound card is controlled by the codec working clock, so the speed is kept consistent. When the amount of audio data written into the cache by the application's playback thread exceeds the end value, the playback thread is controlled to enter a blocked state and stop writing the audio data to be played into the cache. As the physical sound card consumes the audio data to be played in the cache, when the amount of audio data in the cache is less than the end value, the playback thread is awakened.

[0110] The technical solution of the embodiment of the present invention controls the startup of all physical sound cards by setting the minimum value of the cache of the virtual sound card layer, and starts each physical sound card at the same time when the amount of audio data in the cache is greater than the minimum value. This can avoid the situation where, when the central processing unit is highly loaded, the physical sound card has no audio data to read during two adjacent scheduling processes of the playback thread, thereby causing noise.

[0111] Optionally, in one embodiment, the playback thread writes the audio data to be played into the cache of the virtual sound card layer, including: obtaining the write requirement and the preset frame duration; splicing the audio data to be played into each frame data in the virtual sound card layer according to the write requirement, the arrangement of each physical sound card and the frame duration; running the playback thread to write each frame data into the cache of the virtual sound card layer.

[0112] The writing requirements are determined by the product requirements, including the requirements of which channels to play the audio data from. Each frame of data refers to the amount of data written to the virtual sound card layer at one time.

[0113] In this embodiment, exemplarily, the calculation method of the data amount of one frame for each channel in the virtual sound card layer can be expressed as (sampling rate / 1000)×frame duration×bit width. Expectedly, the sampling rate is 48k, the bit width is 16 bits, and the frame duration is 5ms. Then the number of frames for each channel is 48000 / 1000×5×(16 / 8)=480 (byte), that is, the data amount of each channel is 480 bytes. Assuming that the virtual sound card layer contains a total of 8 channels, the data amount of the 8 channels is 8×480=3840 bytes of data. These data are spliced ​​into frame data in the virtual sound card layer, that is, the amount of data written to the virtual sound card layer by the application at one time is 3840 bytes.

[0114] Specifically, after the virtual sound card layer is built, the number of channels contained in the virtual sound card layer and the arrangement of each channel can be obtained. Assuming that the virtual sound card layer includes 8 channels, if it is required to play data from the 1st and 5th channels, the application fills the 1st and 5th positions of the cache with a frame of audio data to be played, and fills the other positions with 0 (all data is 0, which means silence). Filling 0 is because data is required when the virtual sound card layer transmits data from I2S to the physical sound card. After receiving the data, the virtual sound card layer distributes the data according to the arrangement of the physical sound card layer and the corresponding number of channels, such as forwarding the data of the 1st to 4th channels to the first physical sound card, the data of the 5th to 6th channels to the second physical sound card, and the data of the 7th to 8th channels to the third physical sound card. In fact, the device corresponding to the first channel of the first physical sound card and the device corresponding to the first channel of the second physical sound card are used to realize the sound output.

[0115] The technical solution of the embodiment of the present invention arranges the data corresponding to each physical channel in the frame data written into the virtual sound card layer, and fills the channel data that does not need to actually play the data with silence, so that each physical sound card can directly read each frame data when reading data from the cache of the virtual sound card layer, and then accurately distributes the data corresponding to each channel in each frame data to the channel of each physical sound card for playback according to the arrangement of the physical sound cards, thereby improving the data distribution efficiency and accuracy, and avoiding the situation where channel distribution errors are easily generated when data is only distributed to the channel that actually plays the data.

[0116] Embodiment 2

[0117] Figure 2 1 is a schematic diagram of the structure of a multi-physical sound card audio data playback system in an embodiment of the present invention. This embodiment is applicable to the case where multiple physical sound card devices play audio data from the same sound source in a synchronous manner. The system can be integrated in any device that provides the function of playing audio data from multiple physical sound cards, such as Figure 2 As shown, the multi-physical sound card audio data playback system specifically includes: a virtual sound card layer 21, a crystal oscillator 22 and at least two physical sound cards 23 (the figure takes two physical sound cards, physical sound card 1 and physical sound card 2, as examples for description).

[0118] Each physical sound card 23 includes a multimedia digital signal codec 231 ; each multimedia digital signal codec 231 is connected to the crystal oscillator 22 .

[0119] The virtual sound card layer 21 is used to cache the audio data to be played written by the playback thread within a preset cache capacity interval;

[0120] The multimedia digital signal codec 231 is used to read the audio data to be played in the cache of the virtual sound card layer according to the clock cycle corresponding to the working clock and play it, wherein the working clock of the multimedia digital signal codec 231 in each physical sound card 23 is obtained by dividing the main clock provided by the crystal oscillator 22 to each physical sound card 23;

[0121] This is to control the difference between the amount of audio data written into the cache by the playback thread and the amount of audio data read from the cache based on the clock cycle to not exceed a preset threshold.

[0122] As an exemplary description of an embodiment of the present invention, Figure 3 FIG. 1 is a schematic diagram of the structure of an exemplary multi-physical sound card audio data playback system in an embodiment of the present invention. Figure 3 As shown, multiple physical sound cards include a crystal oscillator and at least two physical sound cards (physical sound card 1 and physical sound card 2 are used as examples in the figure), and a single physical sound card includes: codec (i.e. multimedia digital signal codec), I2C (Inter-Integrated Circuit, integrated circuit bus), I2S bus (Inter-IC Sound, integrated circuit built-in audio bus) and I2S controller inside the CPU side (not shown in the figure), as well as physical sound card drivers at the software level (sound card 1 driver and sound card 2 driver are used as examples in the figure). The virtual sound card layer inside the CPU is a pure software abstraction layer.

[0123] In the hardware design of multiple physical sound cards, the working clock of the codec of each physical sound card is obtained by dividing the main clock provided by the same crystal oscillator. Each physical sound card uses the codec as the I2S bus master device and the CPU as the I2S bus slave device (that is, the clock when the CPU and the codec transmit data bidirectionally through the I2S bus is provided by the codec, and the working clock of the codec is obtained by dividing the crystal oscillator. Since the working clocks of the codecs of all physical sound cards come from the same crystal oscillator, the clocks sent by the codecs of each physical sound card to the CPU are synchronized).

[0124] Based on the above Figure 3 The following describes the data flow process in which the playback thread and the acquisition thread participate in the various components of the multi-physical sound card audio data playback system.

[0125] As an exemplary description of an embodiment of the present invention, Figure 4 FIG. 1 is a schematic diagram of a data flow process in which a playback thread participates in an embodiment of the present invention. Figure 4As shown in the figure, the data flow process in which the playback thread participates can be described as: the audio data decoded from the network and collected locally is input into the audio mixer, and then output to buf (buffer). The application starts the playback thread to read the audio data to be played from the buf of the mixing output, and writes the read audio data to be played into the virtual sound card layer cache. The DMA in the physical sound card driver starts from the moment the physical sound card is started, and continuously moves audio data from the virtual sound card layer cache to the I2S controller FIFO (First-In, First-Out abbreviation, first-in, first-out, is a data structure or storage management strategy used to organize and access data in a specific order when processing data streams). Then the audio data in the FIFO will continuously move the audio data data to the I2S bus with the working clock provided by the codec, thereby entering the codec and finally playing it out from the codec.

[0126] Among them, DMA refers to a hardware functional module, which is mainly used to move data inside the CPU without the direct involvement of the CPU. After the parameters are configured, it can work uninterruptedly to move data from one location to another. The scenario of this embodiment is to move data from the cache of the virtual sound card layer to the FIFO of the I2S controller. When the FIFO is full, the data movement stops; when half of the FIFO data is consumed, DMA starts to move data again, and so on and so forth. Therefore, the rhythm of DMA moving data is strongly related to the working clock issued by the codec.

[0127] Because the codec output clocks of multiple physical sound cards are synchronized, the rhythm of moving data out from the FIFO of the I2S controller is synchronized. This rhythm controls the rhythm of DMA moving data from the cache of the virtual sound card layer, so the rhythm of data consumption in the cache of the virtual sound card layer is consistent. Therefore, it is only necessary to control the amount of audio data in the cache of the virtual sound card layer between [start, end], and then the running rhythm of the playback thread is reversely controlled. When the speed at which the playback thread writes data is greater than the DMA transfer speed, that is, when the speed at which the playback thread writes data is greater than the actual playback speed of the physical sound card, the amount of audio data in the cache will exceed the value of end. At this time, the virtual sound card layer will control the playback thread to enter a sleep state. When the amount of audio data in the cache of the virtual sound card layer is less than end, the playback thread will be awakened to continue writing the audio data to be played. The setting of another start boundary value is first used to avoid noise caused by no audio data in the cache for the physical sound card to read when the CPU is under high load. The physical sound card is started only after the amount of data written to the cache by the playback thread is greater than the start value for the first time; secondly, after the physical sound card is started, when the amount of data in the cache is less than the start value, it indicates that the writing speed of the playback thread is less than the actual playback speed of the physical sound card (in theory, the physical sound card is started after the amount of audio data in the cache is greater than the start value. If the amount of audio data written to the cache by the playback thread is consistent with the amount of audio data read from the cache by the physical sound card, then the amount of data in the cache will not be less than the start value. Only when the amount of data read from the cache is greater than the amount of data written by the playback thread will the amount of audio data in the cache be less than the start value). It is necessary to increase the speed at which the playback thread sends data through the acquisition rhythm control of the acquisition thread. The specific control logic has been introduced in detail above and will not be repeated here.

[0128] As an exemplary description of an embodiment of the present invention, Figure 5 Schematic diagram of a data flow process in which a collection thread participates in an embodiment of the present invention. The collection thread reads data by calling the read interface of the virtual sound card layer, such as Figure 5As shown, the data flow process in which the acquisition thread participates can be described as follows: the target data data is shifted into the I2S controller FIFO along with the codec's working clock clock, and the physical sound card driver reads the target data from the I2S controller FIFO into the virtual sound card layer cache, and the reading rhythm is controlled by the codec's working clock clock. When the acquisition thread reads the target data from the virtual sound card layer's cache, it will return if there is data, and the thread will be blocked if there is no data. Therefore, the acquisition thread's working rhythm is controlled by the codec's working clock clock. The acquisition thread's working rhythm is the rhythm of the codec's playback or acquisition. The acquisition thread is used to adjust the rhythm of the application-state playback thread to prevent the playback thread from running too slowly. It should be noted that the caches used to store target data and audio data to be played in the virtual sound card layer are different spaces.

[0129] The technical solution of the embodiment of the present invention is to build a virtual sound card layer at the software level, connect the multimedia digital signal codec codec of each physical sound card to the same crystal oscillator in hardware, and use the same crystal oscillator to provide a master clock for each codec and obtain the working clock corresponding to each codec after frequency division, so that each codec reads the audio data to be played in the cache of the virtual sound card layer according to the clock cycle corresponding to the working clock and plays it, ensuring the synchronization of each physical sound card playback. In addition, the cache capacity interval for storing the audio data to be played in the virtual sound card layer is preset, and the amount of audio data written into the cache by the control playback thread based on the working clock and the preset cache capacity interval does not exceed the preset threshold value, so that a certain amount of audio data to be played can be stored in the cache of the virtual sound card layer for the physical sound card to read, and there will be no delay caused by too much cache data or noise caused by too little cache data. Through the technical solution of the present invention, the problem of asynchronism when multiple sound card devices play the same source sound, the problem of too long delay, and the problem of noise in some sound card playback devices caused by rhythm problems can be solved.

[0130] Embodiment 3

[0131] Figure 6A schematic diagram of the structure of an electronic device 30 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile systems, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing systems. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention application described and / or required herein.

[0132] like Figure 6 As shown, the electronic device 30 includes at least one processor 31, and a memory connected to the at least one processor 31 in communication, such as a read-only memory (ROM) 32, a random access memory (RAM) 33, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 31 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 32 or the computer program loaded from the storage unit 38 to the random access memory (RAM) 33. In the RAM 33, various programs and data required for the operation of the electronic device 30 can also be stored. The processor 31, the ROM 32, and the RAM 33 are connected to each other through a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0133] A number of components in the electronic device 30 are connected to the I / O interface 35, including: an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a disk, an optical disk, etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 39 allows the electronic device 30 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0134] The processor 31 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 31 performs the various methods and processes described above, such as a multi-physical sound card audio data playback method:

[0135] The playback thread writes the audio data to be played into the cache of the virtual sound card layer;

[0136] The multimedia digital signal codec in each physical sound card obtains the audio data to be played in the buffer and plays it according to the clock cycle corresponding to the working clock, wherein the working clock is obtained by dividing the main clock provided by the same crystal oscillator to each physical sound card;

[0137] Based on the working clock and the preset cache capacity interval of the virtual sound card layer, the error between the amount of audio data written into the cache by the playback thread and the amount of audio data read from the cache based on the clock cycle does not exceed a preset threshold.

[0138] In some embodiments, the multi-physical sound card audio data playing method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 38. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 30 via the ROM 32 and / or the communication unit 39. When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the multi-physical sound card audio data playing method described above can be performed. Alternatively, in other embodiments, the processor 31 can be configured to execute the multi-physical sound card audio data playing method in any other appropriate manner (for example, by means of firmware).

[0139] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0141] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0142] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0143] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0144] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0145] In one embodiment, the embodiment of the present invention further includes a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, the multi-physical sound card audio data playback method of any embodiment of the present invention is implemented.

[0146] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0147] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0148] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for playing audio data of multiple physical sound cards, characterized in that: include: The playback thread writes the audio data to be played into the cache of the virtual sound card layer; The multimedia digital signal codec in each of the physical sound cards obtains the audio data to be played in the cache and plays it according to the clock cycle corresponding to the working clock, wherein the working clock is obtained by dividing the main clock provided by the same crystal oscillator to each of the physical sound cards; Based on the working clock and the preset cache capacity interval of the virtual sound card layer, the amount of audio data written into the cache by the playback thread and the amount of audio data read from the cache by the physical sound card driver based on the clock cycle are controlled so that the error does not exceed a preset threshold, wherein the physical sound card includes the physical sound card driver.

2. The method according to claim 1, characterized in that The method of controlling the amount of audio data written into the cache by the playback thread based on the working clock and the preset cache capacity interval of the virtual sound card layer and the amount of audio data read from the cache based on the clock cycle to not exceed a preset threshold value includes: Based on the clock cycle of the working clock, the target data in the physical sound card is transferred to the virtual sound card layer; The acquisition thread reads the target data from the virtual sound card layer, and determines the number of first frames of the target data accumulated by the acquisition thread up to the current statistical time; When the playback thread starts the current running round, the number of second frames of the to-be-played audio data accumulated by the playback thread in the cache up to the current statistical time is obtained; Determining the initial difference frame quantity based on the start time of the acquisition thread and the playback thread; When the third frame number obtained by subtracting the initial difference frame number from the second frame number is less than the first frame number, the target frame number to be written into the cache in the current running round of the playback thread is determined based on the first frame number, the third frame number and the cache capacity interval.

3. The method according to claim 2, characterized in that The virtual sound card layer is pre-constructed based on the number of the physical sound cards, the number of channels of each of the physical sound cards, and a preset arrangement of each of the physical sound cards; Before the playing thread writes the to-be-played audio data into the cache of the virtual sound card layer, the method further includes: The attribute information of the virtual sound card layer is configured, and the attribute information of the virtual sound card layer is synchronized to each of the physical sound cards, wherein the attribute information includes a sampling rate, a sampling bit width, and a cache capacity interval.

4. The method according to claim 3, characterized in that The determining, based on the first frame number, the third frame number, and the cache capacity interval, a target frame number written into the cache in the current running round of the playback thread includes: Subtract the first frame quantity from the third frame quantity to obtain a difference frame quantity; Determine the capacity corresponding to the number of difference frames according to the number of difference frames, the preset frame duration, the sampling rate, the sampling bit width, and the number of channels of the virtual sound card layer; Obtaining a remaining cache capacity in the cache based on the cache capacity interval; If the remaining cache capacity in the cache is greater than or equal to the capacity corresponding to the number of difference frames, determining that the target number of frames written into the cache in the current running round of the playback thread is the number of difference frames; If the remaining cache capacity in the cache is less than the capacity corresponding to the number of difference frames, the target number of frames written into the cache in the current running round of the playback thread is determined to be the number of frames corresponding to the remaining cache capacity in the cache, and the number of frames corresponding to the remaining cache capacity in the cache is determined by the remaining cache capacity in the cache, the frame duration, the sampling rate, the sampling bit width and the number of channels of the virtual sound card layer.

5. The method according to claim 1, characterized in that Based on the working clock and the preset cache capacity interval of the virtual sound card layer, the error between the amount of audio data written into the cache by the playback thread and the amount of audio data read from the cache based on the clock cycle does not exceed a preset threshold, including: When the amount of audio data written into the cache in the current running round of the playback thread is greater than the maximum value of the cache capacity interval, the playback thread is controlled to enter a blocked state and stop writing the audio data to be played into the cache until the amount of audio data in the cache is less than the maximum value of the cache capacity interval, and then the playback thread is awakened to enter the next running round.

6. The method according to claim 1, characterized in that The cache capacity interval is determined by a minimum value and a maximum value, wherein the minimum value is the amount of data read by the physical sound card during two adjacent scheduling processes of the playback thread when the CPU is under high load; After the playing thread writes the audio data to be played into the cache of the virtual sound card layer, it also includes: When the amount of audio data in the buffer of the virtual sound card layer is greater than the minimum value, each of the physical sound cards is started simultaneously.

7. The method according to claim 1, characterized in that The playing thread writes the audio data to be played into the cache of the virtual sound card layer, including: Get writing requirements and preset frame duration; splicing the audio data to be played into each frame data in the virtual sound card layer according to the writing requirement, the arrangement mode of each physical sound card and the frame duration; The playing thread is run to write each frame of data into the cache of the virtual sound card layer.

8. A multi-physical sound card audio data playback system, characterized in that: include: A virtual sound card layer, a crystal oscillator, and at least two physical sound cards; The physical sound card comprises: a multimedia digital signal codec; each of the multimedia digital signal codecs is connected to the crystal oscillator; The virtual sound card layer is used to cache the audio data to be played written by the playback thread within a preset cache capacity interval; The multimedia digital signal codec is used to obtain the audio data to be played in the buffer of the virtual sound card layer and play it according to the clock cycle corresponding to the working clock, wherein the working clock is obtained by dividing the main clock provided by the crystal oscillator to each of the physical sound cards; The amount of audio data written into the cache by the playback thread is controlled so that the error between the amount of audio data read from the cache by the physical sound card driver based on the clock cycle does not exceed a preset threshold, wherein the physical sound card includes the physical sound card driver.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the multi-physical sound card audio data playing method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the multi-physical sound card audio data playing method according to any one of claims 1 to 7.

Images