Audio channel load distribution method and system

Abstract

The invention provides an audio channel load distribution method and system. The method comprises the following steps: acquiring audio signals and audio channel load data corresponding to a plurality of audio channels; dividing the plurality of audio channels to obtain a main channel group, an auxiliary channel group and a standby channel group; for any one audio channel, the following steps are executed: obtaining an audio channel gain based on an audio channel load data audio signal, and limiting the channel gain corresponding to the audio channel to a preset gain threshold based on the audio channel gain; and acquiring main channel real-time load data, and when the main channel real-time load data exceeds the first preset load threshold value, distributing the main channel real-time load data until the main channel real-time load data is lower than the second preset load threshold value, thereby realizing balanced distribution of the audio channel load. According to the audio channel load distribution method and system provided by the invention, an audio channel overload phenomenon is avoided, and self-adaptive adjustment of audio channel gains and dynamic allocation of audio channel resources are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of digital audio processing, in particular to a method and system for audio channel load distribution. BACKGROUND

[0002] Multi-channel audio gain adjustment technology has been widely used in digital audio processing, conference systems, live broadcast equipment, sound engineering, etc. However, with the increasing complexity of audio equipment and the diversification of application scenarios, more accurate control and intelligent processing of multi-channel audio signals are required. Traditional multi-channel gain adjustment mainly relies on fixed parameter control, all channels use the same processing strategy, and it is difficult to dynamically optimize according to the differences in audio content and actual application requirements. In recent years, with the rapid development of digital signal processing technology and artificial intelligence algorithms, intelligent and adaptive audio processing technology has gradually become a research hotspot. Currently, there are three main existing technical solutions: one is multi-channel gain control based on AGC, which detects the average power or peak value of the audio signal and automatically adjusts the gain to maintain stable output level; two is dynamic range control based on compressor, which divides the audio signal into multiple frequency bands and independently performs dynamic range compression on each frequency band; three is gain switching based on pre-set scenarios, which pre-sets multiple audio scenarios (such as conference, music, movie, etc.), and switches the corresponding gain parameters according to user selection.

[0003] The existing technology still has obvious limitations: first, there is a lack of intelligent channel priority management - the same processing strategy is used for all channels, which cannot distinguish between important channels and secondary channels, and thus it is difficult to dynamically allocate processing resources according to the importance of audio content, especially in multi-person conference scenarios, it is difficult to highlight the voice of the main speaker; second, the static gain adjustment strategy is inefficient - traditional AGC is based on fixed time constant and threshold, with slow response speed, which cannot make fine adjustments according to the real-time characteristics of the audio signal, resulting in low resource utilization and all channels running at full capacity; and there is a lack of effective load balancing mechanism - when the load of a channel is too high, it is difficult to dynamically allocate resources of other channels, making it difficult to fully utilize the parallel processing capacity of the system, and single-point overload phenomenon is likely to occur, affecting the overall audio quality; finally, the synchronization between channels is poor - each channel processes audio signals independently, lacks unified timing control, and processing delays are inconsistent, which can easily lead to audio phase distortion and seriously affect the spatial positioning effect in multi-channel surround sound applications. SUMMARY

[0004] The present application aims to provide a method and system for audio channel load distribution to solve the above technical problems and avoid audio channel overload, and to achieve adaptive adjustment of audio channel gain and dynamic allocation of audio channel resources.

[0005] To solve the above technical problems, the present application provides a method for audio channel load distribution, comprising the following steps:

[0006] Obtaining audio signals corresponding to a plurality of audio channels and audio channel load data;

[0007] Dividing the plurality of audio channels based on the audio signals to obtain a main channel group, an auxiliary channel group and a backup channel group;

[0008] For any one audio channel, the following steps are performed: based on the audio channel load data corresponding to the audio channel and the audio signal corresponding to the audio channel, obtaining the audio channel gain corresponding to the audio channel, and limiting the channel gain corresponding to the audio channel to a preset gain threshold based on the audio channel gain;

[0009] Obtaining main channel real-time load data corresponding to the main channel group, and when the main channel real-time load data exceeds a first preset load threshold, distributing the main channel real-time load data based on the main channel, the auxiliary channel and the backup channel until the main channel real-time load data is below a second preset load threshold, to achieve balanced distribution of the audio channel load.

[0010] In the above scheme, first, the audio signals are obtained to provide a basis for subsequent division of the audio channels and calculation of the audio channel gain; at the same time, the audio channel load data is collected to provide data support for subsequent balanced distribution of the audio channel load. Then, the audio channels are divided based on the characteristics of the audio signals, so that the main channel that exceeds the first preset load threshold can be precisely distributed. Then, for any one audio channel, the audio channel gain corresponding to the audio channel is obtained in combination with the corresponding load data and the corresponding audio signal, which lays a prerequisite for subsequent distribution of the main channel real-time load data, ensuring that the subsequent load allocation can adapt to the processing requirements of the audio signals and precisely control based on the actual running state of the channel. Finally, when the main channel real-time load data exceeds the first preset load threshold, load redistribution is started to transfer part of the load of the main channel until the main channel load falls below the second preset threshold. This process can effectively avoid the phenomenon of audio channel overload, ensure the stability of the overall audio, and realize adaptive adjustment of the audio channel gain and dynamic allocation of the audio channel resources.

[0011] Further, the dividing of the plurality of audio channels based on the audio signals to obtain the main channel group, the auxiliary channel group and the backup channel group comprises:

[0012] For any one audio channel, the following steps are performed: based on the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a backup channel; until all audio channels are divided;

[0013] The audio channels divided into the main channel form a main channel group.

[0014] The audio channels divided into the auxiliary channel form an auxiliary channel group.

[0015] The audio channels divided into the backup channel form a backup channel group.

[0016] In the above scheme, the signal intensity can be reflected by obtaining the audio channel energy, the importance of the audio signal can be reflected by obtaining the audio signal, and the audio channel weight is calculated in combination with the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel. Abstract audio characteristics are converted into quantifiable numerical values, providing a basis for subsequent audio channel division.

[0017] Further, the above steps are performed for any one audio channel: based on the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a backup channel; including:

[0018] For any one audio channel, the following steps are performed:

[0019] Performing FFT transformation on the audio signal corresponding to the audio channel to obtain the audio characteristics corresponding to the audio channel;

[0020] Based on the audio channel energy corresponding to the audio channel and the audio characteristics corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained;

[0021] Based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a backup channel.

[0022] In the above scheme, by performing FFT transformation on the audio signal corresponding to the audio channel, the deep features of the audio signal can be extracted, and the time domain audio signal is converted into frequency domain features, providing quantification basis for subsequent audio channel weight evaluation. Then, based on the audio channel energy and the audio characteristics, the multi-dimensional feature is quantified to convert the abstract audio importance into a comparable weight value, ensuring the objectivity and accuracy of the audio channel division.

[0023] Further, the above steps are performed for any one audio channel: based on the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a backup channel; including:

[0024] if the audio channel weight is greater than or equal to the first weight threshold, dividing the audio channel into a main channel;

[0025] if the audio channel weight is greater than or equal to the second weight threshold and less than the first weight threshold, dividing the audio channel into an auxiliary channel;

[0026] if the audio channel weight is less than the second weight threshold, dividing the audio channel into a backup channel.

[0027] In the above scheme, by dividing the audio channel with a weight greater than or equal to the first weight threshold into a main channel, the audio signal corresponding to the main channel usually has high energy, stable time domain characteristics and concentrated spectrum, so the highest priority resource is allocated to it as the core content of priority quality guarantee. By dividing the audio channel with a weight greater than or equal to the second weight threshold and less than the first weight threshold into an auxiliary channel, the audio signal of the auxiliary channel is less important than the main channel, and the processing quality and resource efficiency need to be considered, so it is allocated with medium resources. By dividing the audio channel with a weight less than the second weight threshold into a backup channel, the audio signal corresponding to this type of channel is usually weak noise, redundant signal or non-critical background sound, which has less impact on the overall audio quality, so it is only allocated with basic resources and runs with low load at ordinary times.

[0028] Further, for any one audio channel, the following steps are performed: based on the audio channel load data corresponding to the audio channel and the audio signal corresponding to the audio channel, an audio channel gain corresponding to the audio channel is obtained, and the channel gain corresponding to the audio channel is limited to a preset gain threshold based on the audio channel gain; comprising:

[0029] For any one audio channel, the following steps are performed:

[0030] Based on the audio signal corresponding to the audio channel, a time domain adaptive factor corresponding to the audio channel and a frequency domain compensation factor corresponding to the audio channel are obtained;

[0031] Based on the audio channel load data corresponding to the audio channel, a load balancing factor corresponding to the audio channel is obtained;

[0032] Based on the time domain adaptive factor corresponding to the audio channel, the frequency domain compensation factor corresponding to the audio channel and the load balancing factor corresponding to the audio channel, an audio channel gain corresponding to the audio channel is obtained, and the channel gain corresponding to the audio channel is limited to a preset gain threshold based on the audio channel gain.

[0033] In the above scheme, the time domain adaptive factor, the frequency domain compensation factor and the load balancing factor are used, wherein the time domain adaptive factor and the frequency domain compensation factor can ensure that the calculated audio channel gain is adapted to the real-time content of the audio signal; and the load balancing factor can ensure that the audio channel gain will not exceed the processing capability of the device, so as to finally generate an adaptive audio channel gain value which takes into account both the audio content characteristics and the load state.

[0034] Further, for any one audio channel, the following steps are performed: based on the audio channel load data corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel gain corresponding to the audio channel is obtained, and the channel gain corresponding to the audio channel is limited to a preset gain threshold based on the audio channel gain; including:

[0035] If the audio channel gain is greater than a first preset gain threshold, the channel gain corresponding to the audio channel is determined as the first preset gain threshold;

[0036] If the audio channel gain is less than a second preset gain threshold, the channel gain corresponding to the audio channel is determined as the second preset gain threshold.

[0037] In the above scheme, for the audio channel gain greater than the first preset gain threshold, since too high gain will cause audio signal clipping and high frequency harmonic distortion, at this time the channel gain corresponding to the audio channel is determined as the first preset gain threshold, which can prevent signal distortion caused by too high gain; for the audio channel gain less than the second preset gain threshold, the corresponding audio signal will be excessively attenuated, resulting in loss of effective information, at this time the channel gain corresponding to the audio channel is determined as the second preset gain threshold, which can prevent signal loss caused by too low gain.

[0038] Further, it further includes:

[0039] After balancing the distribution of audio channel load, for any one audio channel, the following steps are performed: based on the algorithm execution time corresponding to the audio channel and the buffer delay corresponding to the audio channel, the total processing delay corresponding to the audio channel is calculated;

[0040] Based on the total processing delay corresponding to the audio channel and the total processing delays corresponding to other audio channels, the synchronization delay of the audio channel and other channels is calculated, and the audio channel is time compensated based on the synchronization delay, so that the audio channel and other audio channels are time aligned.

[0041] In the scheme, the actual time consumption of each channel is quantified by calculating the total processing delay corresponding to the audio channel, thereby providing reference data for multi-channel synchronization. By calculating the synchronization delay and performing time compensation, the time difference between the multi-channels can be eliminated, the audio signals can be output simultaneously, the misalignment or echo in hearing can be avoided, the stability of the single channel is ensured, the time misalignment problem of the multi-channels is solved, and the integrity of the sound quality can be ensured.

[0042] The application provides a system for audio channel load distribution, comprising a data acquisition module, a channel classification module, a gain calculation module, a gain constraint module and an equalization scheduling module, specifically comprising:

[0043] The data acquisition module is configured to acquire audio signals and audio channel load data corresponding to a plurality of audio channels.

[0044] The channel classification module is configured to divide the plurality of audio channels based on the audio signals to obtain a main channel group, an auxiliary channel group and a backup channel group.

[0045] The gain calculation module is configured to, for any one audio channel, perform the following steps: based on the audio channel load data corresponding to the audio channel and the audio signal corresponding to the audio channel, acquiring the audio channel gain corresponding to the audio channel, and limiting the channel gain corresponding to the audio channel to a preset gain threshold based on the audio channel gain.

[0046] The equalization scheduling module is configured to acquire main channel real-time load data corresponding to the main channel group, and when the main channel real-time load data exceeds a first preset load threshold, allocate the main channel real-time load data based on the main channel, the auxiliary channel and the backup channel until the main channel real-time load data is lower than a second preset load threshold, thereby achieving equalization distribution of the audio channel load.

[0047] The system for audio channel load distribution provided by the scheme can provide a basis for subsequent audio channel division and audio channel gain calculation by only acquiring audio signals in actual application; meanwhile, audio channel load data is collected to provide data support for subsequent audio channel load balanced distribution. Then, for any audio channel, the corresponding audio channel gain of the audio channel is obtained by combining the corresponding load data and the corresponding audio signal of the audio channel, which lays a prerequisite for subsequent real-time load data distribution of the main channel, ensuring that the subsequent load allocation can not only adapt to the processing requirements of the audio signal, but also achieve accurate regulation and control based on the actual running state of the channel. Under the condition of meeting the above conditions, the audio channels are divided based on the characteristics of the audio signal to facilitate the subsequent accurate load distribution of the main channel exceeding the first preset load threshold. Finally, when the real-time load data of the main channel exceeds the first preset load threshold, load redistribution is started, and part of the load of the main channel is transferred until the load of the main channel falls below the second preset threshold. This process can effectively avoid the phenomenon of audio channel overload, ensure the stability of the overall audio, and realize adaptive adjustment of the audio channel gain and dynamic allocation of the audio channel resources.

[0048] Further, the channel classification module is configured to divide a plurality of audio channels based on the audio signal to obtain a main channel group, an auxiliary channel group, and a standby channel group; and the channel classification module includes:

[0049] For any audio channel, the following steps are performed: based on the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel, or a standby channel; until all audio channels are divided.

[0050] The audio channels divided into the main channels form the main channel group.

[0051] The audio channels divided into the auxiliary channels form the auxiliary channel group.

[0052] The audio channels divided into the standby channels form the standby channel group.

[0053] In the above scheme, the audio channel energy can reflect the intensity of the signal, and the audio signal can reflect the importance of the audio signal. The audio channel weight is calculated by combining the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, which converts abstract audio characteristics into quantifiable values to provide a basis for subsequent audio channel division.

[0054] Further, for any one audio channel, the following steps are performed: based on the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a backup channel; including:

[0055] For any one audio channel, the following steps are performed:

[0056] Performing FFT transformation on the audio signal corresponding to the audio channel to obtain the audio characteristics corresponding to the audio channel;

[0057] Based on the audio channel energy corresponding to the audio channel and the audio characteristics corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained;

[0058] Based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a backup channel.

[0059] In the above scheme, by performing FFT transformation on the audio signal corresponding to the audio channel, the deep features of the audio signal can be extracted, the time domain audio signal is converted into frequency domain features, and quantitative basis is provided for subsequent audio channel weight evaluation. Then, based on the audio channel energy and the audio characteristics, the multi-dimensional feature is quantized to obtain the audio channel weight, the abstract audio importance is converted into a comparable weight value, and the objectivity and accuracy of the audio channel division are ensured. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 A method flow diagram of audio channel load distribution is provided for an embodiment of the present application;

[0061] Figure 2 A system architecture diagram of audio channel load distribution is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0063] The present embodiment provides a method for audio channel load distribution, and the specific steps are described in Figure 1 , including:

[0064] Step S1: obtaining audio signals and audio channel load data corresponding to a plurality of audio channels;

[0065] Step S2: dividing a plurality of audio channels based on the audio signal to obtain a main channel group, an auxiliary channel group and a backup channel group;

[0066] Step S3: for any one audio channel, the following steps are performed: based on the audio channel corresponding to the audio channel load data and the audio signal corresponding to the audio channel, the audio channel gain corresponding to the audio channel is obtained, and the channel gain corresponding to the audio channel is limited to a preset gain threshold based on the audio channel gain;

[0067] Step S4: obtaining the main channel real-time load data corresponding to the main channel group, when the main channel real-time load data exceeds the first preset load threshold, the main channel real-time load data is allocated based on the main channel, the auxiliary channel and the backup channel, until the main channel real-time load data is below the second preset load threshold, realizing the balanced allocation of the audio channel load.

[0068] In this embodiment, first, by obtaining the audio signal (this embodiment has N audio channels, corresponding to N audio signals S1(t), S2(t), …, S n (t), the signal sampling rate is 48kHz, and the quantization bit number is 24bit), the basis is provided for subsequent audio channel division and audio channel gain calculation; At the same time, the audio channel load data is collected, which provides data support for the subsequent balanced allocation of the audio channel load. Then, based on the audio signal characteristics, the audio channels are divided in order to implement accurate load allocation for the main channel exceeding the first preset load threshold. In this embodiment, the first preset load threshold can be 0.8, and the second preset load threshold can be 0.6 (it can also be dynamically adjusted according to the overall audio channel load data, such as increasing the first preset load threshold to 0.85 in the low load period; reducing the first preset load threshold to 0.75 in the high load period). Then, for any one audio channel, the audio channel gain corresponding to the audio channel is obtained in combination with the corresponding load data and the corresponding audio signal, which lays a prerequisite for subsequent allocation of the main channel real-time load data, ensuring that the subsequent load allocation can adapt to the processing requirements of the audio signal and realize accurate control based on the actual running state of the channel. Finally, when the main channel real-time load data exceeds the first preset load threshold, the load redistribution is started, and part of the load of the main channel is transferred until the main channel load is reduced to below the second preset threshold. This process can effectively avoid the phenomenon of audio channel overload, ensure the stability of the overall audio, realize the adaptive adjustment of the audio channel gain and the dynamic allocation of the audio channel resources. In this embodiment, the monitoring frequency of the real-time load data of the audio channel in the normal state is 100ms, and the monitoring frequency in the high load state (when the real-time load data is greater than 0.6) is 20ms. Specifically, the real-time load data formula is Loadi (t) = 0.5 * CPU i + 0.3Memory i + 0.2delay i ; wherein Load i (t) represents real-time load data of the i-th audio channel, CPU i represents processor occupancy rate of the i-th audio channel, Memory i represents memory usage rate of the i-th audio channel, and delay i represents processing delay normalization value of the i-th audio channel. Then, main channel real-time load data corresponding to the main channel group is obtained from real-time load data of all audio channels, and it is determined whether the first preset load threshold is exceeded. If the first preset load threshold is exceeded, balanced distribution of audio channel load is performed.

[0069] In the embodiment, balanced distribution of audio channel load can also use proactive load management based on prediction: using time series prediction algorithm, ARIMA model is called to predict load change: Load i (t + At) = ARIMA (Load i (t)). Thus, resource allocation is performed in advance to avoid load peaks. In the embodiment, balanced distribution of audio channel load can also use task scheduling based on priority queue: audio processing tasks are sorted into different queues according to priority: such as tasks with high priority are sorted into the main channel task group, tasks with medium priority are sorted into the auxiliary channel task, and tasks with low priority are sorted into the standby channel task.

[0070] Further, the audio signal is used to divide a plurality of audio channels to obtain a main channel group, an auxiliary channel group and a standby channel group; comprising:

[0071] For any one audio channel, the following steps are performed: based on the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a standby channel; until all audio channels are divided;

[0072] The audio channels divided into the main channel form the main channel group.

[0073] The audio channels divided into the auxiliary channel form the auxiliary channel group.

[0074] The audio channels divided into the standby channel form the standby channel group.

[0075] In the embodiment, the signal intensity can be reflected by obtaining the audio channel energy, and the corresponding audio channel energy can be obtained by Ei= 1 / N * åN-1 n=0 Xn-1i (t) represents the audio channel energy corresponding to the i-th audio channel at time t, k represents the sample point index, represents the k-th sample point from the current time point t, ranging from 0 to L-1; L represents the sample window length, S i (t-k) represents the sample value of the audio signal corresponding to the i-th audio channel at (t-k); by obtaining the audio signal, the importance of the audio signal can be reflected, and the audio channel weight is calculated in combination with the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, so as to convert the abstract audio characteristics into quantifiable numerical values, and provide a basis for the subsequent audio channel division.

[0076] Further, for any one audio channel, the following steps are performed: based on the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a standby channel; comprising:

[0077] For any one audio channel, the following steps are performed:

[0078] Performing FFT transformation on the audio signal corresponding to the audio channel to obtain the audio characteristics corresponding to the audio channel;

[0079] Based on the audio channel energy corresponding to the audio channel and the audio characteristics corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained;

[0080] Based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a standby channel.

[0081] In this embodiment, by performing 256-point FFT (Fast Fourier Transform) on the audio signal corresponding to the audio channel: X i (f) = FFT[S i (t)], in the formula, X i (f) represents the audio characteristics corresponding to the i-th audio channel, S i(t) represents the sampling value of the audio signal corresponding to the i-th audio channel at t, the spectral characteristics corresponding to the audio channel can be obtained, the deep features of the audio signal are extracted, the time domain audio signal is converted into frequency domain features, and the subsequent audio channel weight evaluation is provided with quantitative basis. Then, based on the audio channel energy and the audio characteristics, the multi-dimensional feature can be quantified to quantify the audio channel weight, and the abstract audio importance can be converted into a comparable weight value to ensure the objectivity and accuracy of the audio channel division. Based on the audio channel energy corresponding to the audio channel and the audio characteristics corresponding to the audio channel, the specific process of obtaining the audio channel weight corresponding to the audio channel is as follows: first, the audio characteristics matrix is generated by the spectral characteristics corresponding to the audio channel and the audio channel energy corresponding to the audio channel: In the formula, A ij represents the energy proportion of the i-th audio channel energy in the j-th frequency band, |X i (f j )| 2 represents the energy of the i-th audio channel energy in the j-th frequency band, ∑ k |X i (f k )| 2 represents the i-th audio channel energy. Then, the audio channel weight of each audio channel is calculated by the audio characteristics matrix: W i =μ·E nrom (i)+ρ·F norm (i)+σ·T i ; in the formula, W i represents the audio channel weight of the i-th audio channel, μ, ρ, σ all represent weight coefficients, and satisfy μ+ρ+σ=1; E nrom (i) represents the normalized i-th audio channel energy, and the total energy of all audio channels is calculated wherein N represents the total number of audio channels, and the i-th audio channel is normalized: and E nrom (i)∈[0,1]; the energy distribution proportion of the frequency band is calculated wherein f j is the center frequency of the j-th frequency band, the spectral spread of the audio signal corresponding to the i-th audio channel is calculated using Shannon entropy wherein M represents the total number of frequency bands, and the spectral spread of the audio signal corresponding to the i-th audio channel is normalized, and F norm (i)∈[0,1], F norm (i) is larger, the spectrum is more dispersed; then the time domain envelope is calculated, wherein W represents the envelope window length; then the envelope change rate is calculated according to the time domain envelope, Finally, the time-domain stability of the audio signal corresponding to the ith audio channel is obtained wherein σΔE is the ΔE in a period of time i The standard deviation of (t) is T i ∈[0,1], T i The greater the value, the more stable the signal. μ, ρ, σ can be automatically adjusted according to the type of audio content: speech mode: by increasing the time-domain factor weight, the FFT delay can be avoided during time-domain processing, which is suitable for the rapid change of speech and improves the response speed; it can focus on tracking the speech envelope, protect the consonant component, and improve the intelligibility of speech; it can reduce frequency-domain calculation, save DSP resources, and reduce computational load. Music mode: by increasing the frequency-domain factor weight, it can accurately control the gain of each frequency band, maintain the balance of tone color, and improve the frequency resolution; it can avoid transient distortion, maintain music dynamics, and protect the dynamic range; it can make the frequency-domain processing maintain phase consistency and stabilize the audio image.

[0082] In this embodiment, the calculation of the corresponding audio channel weight can also use weight prediction based on machine learning, use a neural network model, and predict the channel importance based on the spectral features corresponding to the ith audio channel, the time-domain features corresponding to the ith audio channel, and the historical statistical features corresponding to the ith audio channel; in this embodiment, the calculation of the corresponding audio channel weight can also learn the weight preference through user interaction behavior (volume adjustment, channel selection, etc.).

[0083] In this embodiment, FFT transformation also uses FFTW library optimization algorithm, specifically: first step, create an intelligent FFT calculation scheme, call the plan generation function of FFTW library to formulate the optimal calculation strategy for N-point real-to-complex FFT transformation, and use FFTW_MEASURE mode for intelligent optimization algorithm selection; second step, use FFTW automatic optimization, first automatically test multiple FFT algorithms, including: Cooley-Tukey base 2 algorithm (suitable for power-of-2 point number), Split-radix split base algorithm (mixed base optimization), and Bluestein chirp Z transform (suitable for any point number). Run each algorithm on the current hardware platform, measure the execution time, and select the fastest algorithm as the final implementation scheme to optimize memory access mode and reduce cache misses; third step, use the optimal calculation scheme generated in the first step and directly call the pre-optimized FFT execution function, so that all N-point FFT transformation is completed at once. The FFTW library optimization algorithm used in this embodiment reduces the complexity from O(N 2 ) of naive DFT to strict O(NlogN), optimizing the computational complexity; there are special optimizations for specific N values (especially powers of 2), which realize the adaptive optimization of hardware; it can automatically enable SIMD instruction sets such as SSE and AVX for vectorization acceleration and automatically utilize the instruction set.

[0084] In this embodiment, the time domain calculation also uses the SIMD instruction set to accelerate, specifically: first, initialize the 8-way parallel accumulator: create a 256-bit wide vector register that can store 8 32-bit floating point numbers at the same time, and initialize the accumulator to a full zero vector, ready for 8-way parallel energy calculation; Then load 8 consecutive audio samples from memory into the vector register at once, use the AVX2 multiplication instruction to square 8 samples at the same time, and add 8 square results to the accumulation vector at the same time, process 8 samples each time until there are less than 8 samples left; Then write the 8 parallel results in the 256-bit vector register to a temporary array, use scalar addition to sum and merge the 8 parallel results to get the total accumulation result of all the 8-way parallel processing before; Further, for the remaining number of samples that cannot be divided by 8 in the calculation, use the traditional sample-by-sample processing method to add each remaining sample squared to the total accumulation result; Finally, divide the total accumulation result by the total length of the signal to get the average energy value of the audio signal, and get the normalized energy calculation result. Through 8-way parallel processing, the calculation speed is increased by 8 times, and continuous memory loading is used to reduce memory access delay; At the same time, the AVX2 instruction simultaneously executes multiple identical operations and then accesses memory in sequence, improving cache hit rate.

[0085] Further, the audio channel is divided into a main channel, an auxiliary channel or a backup channel based on the audio channel weight; comprising:

[0086] If the audio channel weight is greater than or equal to the first weight threshold, the audio channel is divided into a main channel;

[0087] If the audio channel weight is greater than or equal to the second weight threshold and less than the first weight threshold, the audio channel is divided into an auxiliary channel;

[0088] If the audio channel weight is less than the second weight threshold, the audio channel is divided into a backup channel.

[0089] In the embodiment, the first weight threshold can be selected as 0.7, and the second weight threshold can be selected as 0.3. By dividing the audio channels with the audio channel weight greater than or equal to the first weight threshold into the main channels, the audio signals corresponding to the main channels usually have high energy, stable time domain characteristics and concentrated spectrum, and therefore the highest priority resources (60% of the DSP / ARM computing resources) are allocated thereto, and the highest processing precision (full-band processing) is set, as the core content of the priority guarantee quality. By dividing the audio channels with the audio channel weight greater than or equal to the second weight threshold and less than the first weight threshold into the auxiliary channels, the audio signals of the auxiliary channels are less important than the main channels, have medium priority (30% of the DSP / ARM computing resources), and need to balance the processing quality and resource efficiency, and therefore medium resources are allocated thereto, and medium processing precision (main frequency band processing) is set. By dividing the audio channels with the audio channel weight less than the second weight threshold into the standby channels, the standby channels have the lowest priority (10% of the DSP / ARM computing resources), the audio signals corresponding to the standby channels are usually weak noise, redundant signals or non-critical background sound, and have less impact on the overall audio quality, and therefore only basic resources are allocated thereto, and low load operation is maintained at ordinary times, and basic processing precision (simplified processing or straight-through) is set.

[0090] In this embodiment, predictive load management can also be performed, which can predict future load trends based on historical load data, thereby adjusting resource allocation in advance to avoid load mutations. Specifically, 1. Create a data structure for load history records, containing 5 key information: timestamp: record the specific time point of this measurement; CPU load: current processor usage percentage; memory usage: current memory usage percentage; processing delay: time delay generated by audio processing; active channel number: the number of currently working audio channels. The data storage strategy is: use a circular buffer to store the last 1000 historical records; adopt a ring queue structure, new data covers the oldest data; maintain an index pointer pointing to the location of the latest data, so that the latest historical information can be automatically maintained. 2. Establish a load trend prediction model, using a weighted moving average prediction algorithm: 1) Set four weight values: 50%, 30%, 15%, and 5%, with higher weight for recent data to reflect the importance of the latest trend; lower weight for long-term data to provide historical reference background. 2) Calculate the predicted value: take the last 4 load data in reverse order from the historical buffer, multiply each historical load value by the corresponding weight coefficient, and add all weighted results to get the base prediction value. 3) Correct the trend: call the load change rate analysis function to get the current load trend, and calculate the trend influence according to the prediction time range (milliseconds), and finally add the trend influence to the base prediction value to get the final prediction result. 3. Analyze the load change rate: 1) Calculate the load change rate: select 10 load sampling points in the last 100 milliseconds to ensure that the data time span is sufficient to reflect short-term trends. 2) Traverse the 10 sampling points to calculate the change rate between adjacent points: calculate the time difference (difference between two timestamps) and load difference (difference between two load values), and divide the load difference by the time difference to get the change rate of this segment. 3) Calculate the average slope: sum all adjacent point change rates and divide by the effective calculation times (total sampling points minus 1) to get the overall average change rate, which reflects the overall trend of load change. 4. Adjust the decision according to the predicted future load value: 1) Call the prediction function to get the expected load value in the future 200 milliseconds (select a prediction window of 200 milliseconds to ensure both early response and prediction accuracy). 2) Process high load warning (when the predicted load exceeds 75%) and start the load balancing preparation program, traverse all auxiliary audio channels, reduce the processing quality of auxiliary audio channels to 90% of the original, and send a notification signal to the system's resource scheduling module to prepare for load balancing, while recording the detailed information of the predictive load management start and the predicted load percentage. 3) When the predicted load is less than 40%, call the channel quality recovery function to gradually improve the audio processing quality, and record the relevant information of load reduction and quality recovery; when the predicted load is between 40% and 75%, maintain the current processing quality and resource allocation state, and continue to execute the prediction algorithm, ready to adjust at any time.

[0091] The prediction algorithm used in this embodiment has a reasonable time window selection: 50 milliseconds for short-term prediction for instant response, with an accuracy of about 95%; 200 milliseconds for medium-term prediction for advance adjustment, with an accuracy of about 87%, balancing prediction accuracy and response time; 500 milliseconds for long-term prediction for trend analysis, with an accuracy of about 72%, providing decision reference. Therefore, by selecting a 200-millisecond prediction window, both advance response and prediction accuracy can be achieved. The weight coefficient design in the prediction algorithm used in this embodiment is scientific: 50% weight for the latest data to ensure quick response to the current state, 30% weight for the second-newest data to maintain sensitivity to short-term changes, 15% and 5% weight for the earlier data to avoid excessive influence of historical abnormal values. The threshold setting in the prediction algorithm used in this embodiment is practical: setting a 75% start threshold to leave enough adjustment time before system overload; setting a 40% recovery threshold to avoid system shock caused by frequent switching; setting a threshold difference of 35% to provide a sufficient hysteresis interval to ensure system stability. This embodiment can improve stability by predicting load and adjusting resources in advance (active prevention), and can improve resource utilization by dynamically adjusting resource allocation based on prediction.

[0092] Further, for any one audio channel, the following steps are performed: based on the audio channel load data corresponding to the audio channel and the audio signal corresponding to the audio channel, an audio channel gain corresponding to the audio channel is obtained, and the channel gain corresponding to the audio channel is limited to a preset gain threshold based on the audio channel gain; comprising:

[0093] For any one audio channel, the following steps are performed:

[0094] Based on the audio signal corresponding to the audio channel, a time domain adaptive factor corresponding to the audio channel and a frequency domain compensation factor corresponding to the audio channel are obtained;

[0095] Based on the audio channel load data corresponding to the audio channel, a load balancing factor corresponding to the audio channel is obtained;

[0096] Based on the time domain adaptive factor corresponding to the audio channel, the frequency domain compensation factor corresponding to the audio channel, and the load balancing factor corresponding to the audio channel, an audio channel gain corresponding to the audio channel is obtained, and the channel gain corresponding to the audio channel is limited to a preset gain threshold based on the audio channel gain.

[0097] In the embodiment, a multi-factor adaptive gain algorithm is adopted, and time domain adaptive factor, frequency domain compensation factor and load balancing factor are used. The time domain adaptive factor and the frequency domain compensation factor can ensure that the calculated audio channel gain is adapted to the real-time content of the audio signal. The load balancing factor can ensure that the audio channel gain will not exceed the processing capacity of the device, and finally an adaptive audio channel gain value is generated which takes into account the audio content characteristics and the load state. The specific calculation process is as follows: i (t) = G base × a i (t) x b i (f) x g i (load); wherein G i (t) represents the audio channel gain corresponding to the i-th audio channel, wherein the audio channel gain corresponding to the main channel is 1.2, the audio channel gain corresponding to the auxiliary channel is 1.0, and the audio channel gain corresponding to the standby channel is 0.8, G base represents the preset audio channel basic gain, a i (t) represents the time domain adaptive factor corresponding to the i-th audio channel, A sliding window mean filter is used, wherein M represents the sliding window length, M = 128, S i (t-k) represents the sample value of the audio signal of the i-th audio channel at (t-k) time, the update frequency is every 32 samples (to reduce the calculation burden), and the time domain adaptive factor needs to be normalized after being obtained: a' i (t) = a i (t) / a max , wherein a max represents the maximum amplitude of the audio signal, and the numerical range is: 16-bit audio: 32767, 24-bit audio: 8388607 (2^23-1), 32-bit floating point: 1.0. a max is mainly to prevent numerical overflow and ensure calculation accuracy. In the embodiment, the frequency domain compensation factor is calculated according to the audio characteristics corresponding to the i-th audio channel, and the frequency domain compensation factor is obtained: wherein b i (f) represents the frequency domain compensation factor corresponding to the i-th audio channel, T(f) represents the preset ideal spectrum distribution of the i-th audio channel, and C i (f) represents the actual spectrum distribution of the i-th audio channel. b i (t) > 1 indicates that the frequency band needs to be enhanced, b i(t) < 1 indicates that the frequency band needs to be attenuated, and the frequency is divided into 32 frequency bands of 20Hz-20kHz, which are logarithmically distributed (in the speech mode, the 300-3400Hz speech frequency band needs to be highlighted; in the music mode, the 20Hz-20kHz full frequency band needs to be balanced; in the conference mode, the low-frequency environmental noise needs to be suppressed). Gamma i (load) indicates the load balancing factor corresponding to the i-th audio channel, wherein, load i (load) indicates the audio channel load data corresponding to the i-th audio channel, load max (load) indicates the maximum allowed audio channel load data of the i-th audio channel, which is usually set to 90%, and 0.3 is a load influence coefficient to ensure that the maximum attenuation does not exceed 30%. Finally, gain smoothing processing is performed on the audio channel gain corresponding to the audio channel: G smooth (t) = 0.9 x G smooth (t-1) + 0.1 x G i (t).

[0098] Further, for any one audio channel, the following steps are performed: based on the audio channel load data corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel gain corresponding to the audio channel is obtained, and the channel gain corresponding to the audio channel is limited to a preset gain threshold based on the audio channel gain; comprising:

[0099] If the audio channel gain is greater than a first preset gain threshold, the channel gain corresponding to the audio channel is determined as the first preset gain threshold;

[0100] If the audio channel gain is less than a second preset gain threshold, the channel gain corresponding to the audio channel is determined as the second preset gain threshold.

[0101] In this embodiment, the first preset gain threshold can be 10, and the second preset gain threshold can be 0.1. If the audio channel gain is not within the appropriate threshold range, soft limiting algorithm is used to achieve smooth limiting using the tanh function: if the audio channel gain is greater than the first preset gain threshold, the high gain will cause the audio signal to be clipped and produce high-frequency harmonic distortion, and at this time the first preset gain threshold is used as the audio channel gain corresponding to the audio channel, which can prevent signal distortion caused by excessive gain. When the audio channel gain is less than the second preset gain threshold, the audio signal will be excessively attenuated, resulting in loss of effective information, and at this time the second preset gain threshold is used as the audio channel gain corresponding to the audio channel, which can prevent signal loss caused by excessive gain. By using the tanh mathematical function, smooth transition of the audio channel gain can be achieved, and gradual convergence is achieved near the limiting boundary, avoiding abrupt changes that cause audio distortion, and the naturalness of the audio can be maintained.

[0102] The specific process of implementing smooth limiting by using the tanh function through the soft limiting algorithm in this embodiment is as follows: the initial audio channel gain calculated is received, and the center point of the limiting interval is calculated through the first preset gain threshold and the second preset gain threshold, that is, the first preset gain threshold and the second preset gain threshold are added and then divided by 2 to obtain the center point value of the limiting interval as 5.05, the radius range of the limiting interval is calculated through the first preset gain threshold and the second preset gain threshold, that is, the first preset gain threshold is subtracted from the second preset gain threshold and then divided by 2 to obtain the distance from the center to the boundary as 4.95, the scaling factor is calculated through the radius range, that is, 2 is divided by the radius range to obtain the scaling coefficient for mapping the initial audio channel gain to the appropriate input range of the tanh function, which is used to control the steepness of the transition region of the tanh function. Then the initial audio channel gain is normalized, that is, the initial audio channel gain value is subtracted from the center point value, and then multiplied by the scaling factor, which is used to map the initial audio channel gain value in any range to the effective working interval of the tanh function, and then the tanh function is calculated on the normalized initial audio channel gain value to smoothly limit the output result of the tanh function between -1 and +1. Finally, the output result of the tanh function is multiplied by the radius range, and then added to the center point value to obtain the output value limited in the target range. Moreover, the process of implementing smooth limiting by using the tanh function in this embodiment can also adopt different processing methods according to the position of the input initial audio channel gain value: 1. Processing of the normal working region: the input initial audio channel gain value is between 1.2 times the second preset gain threshold and 0.8 times the maximum value, at this time, the initial audio channel gain value is directly used without any limiting processing, which can avoid unnecessary calculation overhead and maintain the original characteristics of the signal; 2. Processing of the boundary dangerous region: the input initial audio channel gain value is close to or exceeds the limiting boundary, the tanh smooth limiting function is enabled, which can start smooth transition when the limit is about to be exceeded, avoiding hard cutting. The tanh function adopted in this embodiment has S-shaped curve characteristics, which can naturally and smoothly converge near the boundary, thereby avoiding the high-frequency harmonic distortion caused by traditional hard limiting and maintaining the continuity and derivability of the signal. In this embodiment, the segmented strategy is adopted for the initial audio channel gain value to reduce unnecessary calculation amount, that is, the initial audio channel gain value is directly used under normal circumstances with zero delay, and the smooth processing is enabled only when needed, and the calculation complexity is low, which is suitable for real-time audio processing requirements. This embodiment also has the advantage of sound quality protection: the "click" sound and distortion caused by hard limiting are eliminated through the tanh smooth limiting, which can maintain the natural dynamic characteristics of the audio signal, and can still maintain the audible audio output in extreme cases.

[0103] Further, it also includes:

[0104] After the equal distribution of the audio channel load is realized, for any one audio channel, the following steps are executed: based on the algorithm execution time corresponding to the audio channel and the buffer delay corresponding to the audio channel, the total processing delay corresponding to the audio channel is calculated;

[0105] Based on the total processing delay corresponding to the audio channel and the total processing delays corresponding to other audio channels, the synchronization delay of the audio channel and other channels is calculated, and the audio channel is time compensated based on the synchronization delay, so that the audio channel is time aligned with other audio channels.

[0106] In the embodiment, by calculating the total processing delay corresponding to the audio channel, the actual time consumption of each channel is quantified, providing reference data for multi-channel synchronization. By calculating the synchronization delay and performing time compensation, the time difference between multiple channels can be eliminated, ensuring simultaneous output of audio signals and avoiding auditory misalignment or echo, ensuring the stability of a single channel and solving the time misalignment problem of multiple channels, and ensuring the integrity of sound quality.

[0107] In the embodiment, a unified clock reference can also be provided by using a dedicated clock chip, so that each audio channel is strictly synchronized according to the hardware clock; in the embodiment, the synchronization of the audio channels can also be realized based on the distributed synchronization of the network time protocol.

[0108] The application provides an audio channel load distribution system, which comprises a data acquisition module, a channel classification module, a gain calculation module, a gain constraint module and an equalization scheduling module, and specifically comprises:

[0109] The data acquisition module is used to acquire audio signals and audio channel load data corresponding to a plurality of audio channels;

[0110] The channel classification module is used to divide a plurality of audio channels based on the audio signals, to obtain a main channel group, an auxiliary channel group and a backup channel group;

[0111] The gain calculation module is used to, for any one audio channel, execute the following steps: based on the audio channel load data corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel gain corresponding to the audio channel is acquired, and the channel gain corresponding to the audio channel is limited to a preset gain threshold based on the audio channel gain;

[0112] The equalization scheduling module is used to acquire main channel real-time load data corresponding to the main channel group, and when the main channel real-time load data exceeds a first preset load threshold, the main channel real-time load data is distributed based on the main channel, the auxiliary channel and the backup channel, until the main channel real-time load data is lower than a second preset load threshold, so that the equal distribution of the audio channel load is realized.

[0113] The system for audio channel load distribution provided by the embodiment only needs to obtain audio signals in actual application to provide basis for subsequent division of audio channels and calculation of audio channel gains; meanwhile, audio channel load data is collected to provide data support for subsequent equal load distribution of audio channels. Then, for any audio channel, the corresponding audio channel gain of the audio channel is obtained by combining the corresponding load data and the corresponding audio signal, which lays a prerequisite for subsequent load data distribution of the main channel, ensuring that the subsequent load distribution can not only adapt to the processing requirements of the audio signal, but also achieve accurate regulation and control based on the actual running state of the channel. Under the condition of meeting the above conditions, the audio channels are divided based on the characteristics of the audio signal to facilitate the subsequent accurate load distribution of the main channel exceeding the first preset load threshold. Finally, when the real-time load data of the main channel exceeds the first preset load threshold, load redistribution is started, and part of the load of the main channel is transferred until the load of the main channel decreases to below the second preset threshold. This process can effectively avoid the phenomenon of overload of the audio channel, ensure the stability of the overall audio, and realize adaptive adjustment of the audio channel gain and dynamic distribution of the audio channel resources. The embodiment can make the audio channel gain transition smoothly, control quality fluctuations, and improve the stability of audio quality.

[0114] The specific process of equal load distribution of the audio channel of the embodiment is as follows: the real-time load data of the main channel exceeds the first preset load threshold, and a load too high alarm is issued: the alarm time and the real-time load value are recorded, and the management module in the system is notified. Then, the standby channel is activated: the standby channel in the standby state is activated, and the basic computing resources are distributed. Then, task redistribution is performed: the processing quality of all auxiliary channels is reduced to 80% of the original (the FFT point number is reduced and the frequency band division is reduced), the parameter update frequency of the auxiliary channel is reduced by half (for example, the original update frequency is every 5 ms, after adjustment, it becomes every 10 ms, thereby the number of calculation calls can be reduced by 50%), the computing resources are released, and the bypass mode of all standby channels is enabled, so that the audio signal is directly transmitted without any processing, thereby the released resources are distributed to the main channel. Finally, the real-time load data of the main channel is recalculated, and when the load of the main channel decreases to below the second preset threshold, the distribution is stopped, and the resource distribution table is updated, and the load balancing log is recorded.

[0115] Further, the channel classification module is configured to divide a plurality of audio channels based on the audio signal to obtain a main channel group, an auxiliary channel group, and a standby channel group; and includes:

[0116] For any one audio channel, the following steps are performed: based on the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a backup channel; until all audio channels are divided;

[0117] The audio channels divided into the main channel form a main channel group.

[0118] The audio channels divided into the auxiliary channel form an auxiliary channel group.

[0119] The audio channels divided into the backup channel form a backup channel group.

[0120] In this embodiment, the signal intensity can be reflected by obtaining the audio channel energy, the importance of the audio signal can be reflected by obtaining the audio signal, and the audio channel weight is calculated in combination with the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, so as to convert the abstract audio characteristics into quantifiable values and provide a basis for subsequent audio channel division.

[0121] Further, for any one audio channel, the following steps are performed: based on the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a backup channel; including:

[0122] For any one audio channel, the following steps are performed:

[0123] Performing FFT transformation on the audio signal corresponding to the audio channel to obtain the audio characteristics corresponding to the audio channel;

[0124] Based on the audio channel energy corresponding to the audio channel and the audio characteristics corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained;

[0125] Based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a backup channel.

[0126] In this embodiment, by performing FFT transformation on the audio signal corresponding to the audio channel, the deep features of the audio signal can be extracted, the time domain audio signal is converted into frequency domain features, and a quantification basis is provided for subsequent audio channel weight evaluation. Then, based on the audio channel energy and the audio characteristics, the multi-dimensional feature is quantified to convert the abstract audio importance into a comparable weight value, so as to ensure the objectivity and accuracy of the audio channel division.

[0127] To evaluate the performance improvement of FFTW+SIMD compared with standard C implementation and FFTW optimization, the following examples are used, as shown in the following table:

[0128] Computational method 1024 point FFT time Energy computation time Speedup Standard C implementation 120us 15us 1.0x FFTW optimized 25us 15us 4.8x FFTW + SIMD 25us 2us 7.5x

[0129] The performance improvement of FFTW+SIMD in this embodiment: the time domain energy calculation speed is improved by 7.5 times, and the overall FFT+time domain processing is improved by 4.8 times.

[0130] The method and system for audio channel load distribution provided in this embodiment optimize the response from seconds to milliseconds, improving the response speed; enhance adaptability, from fixed parameter control to multi-factor adaptive control; through intelligent resource allocation, unnecessary calculation overhead is avoided, and processing efficiency is improved; the signal-to-noise ratio is improved, the total harmonic distortion is reduced, and the audio quality is improved; the end-to-end delay control is within 5ms, meeting the real-time application requirements and ensuring real-time performance. Compared with the scene-based audio processing, the embodiment does not need manual scene setting, can automatically identify and adapt to audio characteristics, and has high intelligence; can support continuous dynamic adjustment, is not limited to discrete preset mode, and has strong flexibility; the single algorithm used in the embodiment is suitable for various application scenarios, and has good universality. The embodiment has advantages in engineering practicability: through dynamic load balancing, system resources are effectively utilized, and the resource utilization rate is high; can support multi-channel audio processing, can be flexibly configured according to requirements, and has strong expansibility.

[0131] The above is the preferred embodiment of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, several improvements and refinements can be made, which are also considered within the scope of protection of the present application.

Claims

1. A method of audio channel load distribution, characterized by, The method comprises the following steps: Obtaining audio signals corresponding to a plurality of audio channels and audio channel load data; Dividing the plurality of audio channels based on the audio signals to obtain a main channel group, an auxiliary channel group and a backup channel group; For any one audio channel, the following steps are performed: based on the audio channel load data corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel gain corresponding to the audio channel is obtained, and the channel gain corresponding to the audio channel is limited to a preset gain threshold based on the audio channel gain; Obtaining main channel real-time load data corresponding to the main channel group, and when the main channel real-time load data exceeds a first preset load threshold, the main channel real-time load data is allocated based on the main channel, the auxiliary channel and the backup channel until the main channel real-time load data is lower than a second preset load threshold, thereby achieving balanced allocation of audio channel load.

2. A method of audio channel load distribution as claimed in claim 1, wherein, The method comprises the following steps: For any one audio channel, the following steps are performed: based on the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; the audio channel is divided into a main channel, an auxiliary channel or a backup channel based on the audio channel weight; until all audio channels are divided; The audio channels divided into the main channel form the main channel group; The audio channels divided into the auxiliary channel form the auxiliary channel group; The audio channels divided into the backup channel form the backup channel group.

3. A method of audio channel load distribution according to claim 2, characterized in that, The method comprises the following steps: For any one audio channel, the following steps are performed: Performing FFT transformation on the audio signal corresponding to the audio channel to obtain the audio characteristics corresponding to the audio channel; Based on the audio channel energy corresponding to the audio channel and the audio characteristics corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; The audio channel is divided into a main channel, an auxiliary channel or a backup channel based on the audio channel weight. The method comprises the following steps:

4. A method of audio channel load distribution according to claim 3, characterized in that, If the audio channel weight is greater than or equal to a first weight threshold, the audio channel is divided into a main channel; If the audio channel weight is greater than or equal to a second weight threshold and less than the first weight threshold, the audio channel is divided into an auxiliary channel; If the audio channel weight is less than the second weight threshold, the audio channel is divided into a backup channel. The method comprises the following steps:

5. The method of audio channel load distribution according to claim 1, wherein, For any one audio channel, the following steps are performed: based on the audio channel load data corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel gain corresponding to the audio channel is obtained, and the channel gain corresponding to the audio channel is limited to a preset gain threshold based on the audio channel gain; ​ Based on the audio signal corresponding to the audio channel, the time domain adaptive factor corresponding to the audio channel and the frequency domain compensation factor corresponding to the audio channel are obtained; Based on the audio channel load data corresponding to the audio channel, the load balancing factor corresponding to the audio channel is obtained; Based on the time domain adaptive factor corresponding to the audio channel, the frequency domain compensation factor corresponding to the audio channel and the load balancing factor corresponding to the audio channel, the audio channel gain corresponding to the audio channel is obtained, and the channel gain corresponding to the audio channel is limited to a preset gain threshold based on the audio channel gain.

6. The method of audio channel load distribution according to claim 1, wherein, The audio channel gain corresponding to the audio channel is obtained based on the audio channel load data corresponding to the audio channel and the audio signal corresponding to the audio channel, and the channel gain corresponding to the audio channel is limited to a preset gain threshold based on the audio channel gain. If the audio channel gain is greater than a first preset gain threshold, the channel gain corresponding to the audio channel is determined as the first preset gain threshold; If the audio channel gain is less than a second preset gain threshold, the channel gain corresponding to the audio channel is determined as the second preset gain threshold.

7. The method of audio channel load distribution according to claim 1, wherein, Further comprising: After achieving balanced allocation of audio channel load, for any one audio channel, the following steps are performed: based on the algorithm execution time corresponding to the audio channel and the buffer delay corresponding to the audio channel, the total processing delay corresponding to the audio channel is calculated; Based on the total processing delay corresponding to the audio channel and the total processing delay corresponding to other audio channels, the synchronization delay of the audio channel and other channels is calculated, and the audio channel is time compensated based on the synchronization delay, so that the audio channel is time aligned with other audio channels.

8. A system for audio channel load distribution, characterized by It comprises a data acquisition module, a channel classification module, a gain calculation module, a gain constraint module and an equalization scheduling module, specifically: The data acquisition module is used to obtain audio signals and audio channel load data corresponding to a plurality of audio channels; The channel classification module is used to divide a plurality of audio channels based on the audio signals to obtain a main channel group, an auxiliary channel group and a standby channel group; The gain calculation module is used to obtain the audio channel gain corresponding to any one audio channel based on the audio channel load data corresponding to the audio channel and the audio signal corresponding to the audio channel, and limit the channel gain corresponding to the audio channel to a preset gain threshold based on the audio channel gain. The equalization scheduling module is used to obtain main channel real-time load data corresponding to the main channel group, and when the main channel real-time load data exceeds a first preset load threshold, the main channel real-time load data is allocated based on the main channel, the auxiliary channel and the standby channel, until the main channel real-time load data is lower than a second preset load threshold, achieving balanced allocation of audio channel load.

9. A system for audio channel load distribution according to claim 8, characterized in that, The channel classification module is used to divide a plurality of audio channels based on the audio signals to obtain a main channel group, an auxiliary channel group and a standby channel group; comprising: For any one audio channel, the following steps are performed: based on the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; based on the audio channel weight, the audio channel is divided into a main channel, an auxiliary channel or a backup channel; until all audio channels are divided; The audio channels divided into the main channel form a main channel group; The audio channels divided into the auxiliary channel form an auxiliary channel group; The audio channels divided into the backup channel form a backup channel group.

10. A system for audio channel load distribution according to claim 9, characterized in that, The audio channel weight corresponding to the audio channel is obtained based on the audio channel energy corresponding to the audio channel and the audio signal corresponding to the audio channel; The audio channel is divided into a main channel, an auxiliary channel or a backup channel based on the audio channel weight; comprising: For any one audio channel, the following steps are performed: Performing FFT transformation on the audio signal corresponding to the audio channel to obtain the audio characteristics corresponding to the audio channel; Based on the audio channel energy corresponding to the audio channel and the audio characteristics corresponding to the audio channel, the audio channel weight corresponding to the audio channel is obtained; The audio channel is divided into a main channel, an auxiliary channel or a backup channel based on the audio channel weight.

Images