Multi-channel sound mixing processing method and system
By using dynamic phase alignment, multi-dimensional frequency deconstruction, and energy conservation field gain adjustment, combined with an intelligent impulse response database, the problems of sound image shift and unreasonable energy distribution in multi-channel mixing are solved, achieving accurate sound source localization and improved auditory immersion.
Patent Information
- Application Number
- CN202511870703.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-12
- Publication Date
- 2026-03-17
AI Technical Summary
Existing multi-channel mixing methods cannot dynamically adjust the delay between channels in real time, resulting in sound image shift and unreasonable energy distribution. They also lack independent separation of the fundamental frequency volume, formant matrix, and transient profile, making it impossible to achieve a precise correlation between sound source location and auditory perception.
By using dynamic phase alignment, multi-dimensional frequency deconstruction, time-frequency unit to three-dimensional spatial coordinate transformation and piecewise dynamic gain adjustment of energy conservation field, combined with dynamic convolution processing using intelligent impulse response database, real-time generation of multi-dimensional mixing parameters is achieved.
It significantly improves the sound source localization accuracy and auditory immersion of multi-channel mixing, ensures the rationality of energy distribution and dynamic range, and enhances the controllability of audio processing and the realism of spatial audio.
Smart Images
Figure CN121687079A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of multi-channel mixing processing technology, specifically referring to a multi-channel mixing processing method and system. Background Technology
[0002] With the development of immersive audio technology, users have placed higher demands on the sound field positioning accuracy, spatial coherence, and auditory comfort of multi-channel mixing.
[0003] However, existing multi-channel mixing processing methods still have certain shortcomings. Most existing technologies use static delay calibration or simple cross-correlation analysis, which cannot dynamically adjust the delay between channels in real time. This leads to sound image shift or low-frequency energy attenuation in multi-channel audio due to phase mismatch. They rely on short-time Fourier transform for spectrum analysis, but lack independent separation of the fundamental frequency volume, formant matrix and transient profile. They generally lack a three-dimensional coordinate mapping mechanism based on the physical audible range, and the sound source location is disconnected from auditory perception. They use global gain adjustment or fixed region division, but do not dynamically adjust the gain coefficient according to acoustic characteristics, resulting in unreasonable energy distribution. Therefore, a multi-channel mixing processing method and system are proposed. Summary of the Invention
[0004] The purpose of this invention is to provide a multi-channel mixing processing method and system to solve the problems mentioned in the background art.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a multi-channel mixing processing method, comprising the following steps: S1. Based on the original multi-channel audio, perform millisecond-level delay calibration between channels through dynamic phase alignment; S2. Based on the phase-aligned channel signal, the fundamental frequency volume, formant matrix, and transient profile are separated using a multi-dimensional frequency deconstruction method. S3. Based on the separated spectral components, perform the transformation from time-frequency units to three-dimensional spatial coordinates; S4. Based on the three-dimensional spatial coordinate distribution, perform piecewise dynamic gain adjustment of the energy conservation field; S5. The signal after gain adjustment is dynamically convolved through an intelligently matched impulse response database. S6. Integrate all preceding processed data to generate multi-dimensional mixing parameters in real time.
[0006] Preferably, in step S1, multi-channel audio capture is initiated via the audio acquisition API to capture raw audio data. Preliminary preprocessing of the captured raw data includes normalization, baseline correction, and elimination of initial noise interference. One channel is selected as a reference channel, and the phase difference between other channels and the reference channel is calculated through cross-correlation analysis. This process is as follows: In the formula, This represents the weighted total phase difference between the target channel and the reference channel. This represents the instantaneous energy of the i-th frequency component. This represents the phase angle of the reference channel at the i-th frequency component. Let N represent the phase angle of the target channel at the i-th frequency component, and N represent the total number of frequency components. The phase difference is converted into a time delay, implemented as follows: f represents frequency; Based on the phase difference calculation results, the amount of delay that needs to be adjusted for each channel is determined. Precise delay at the millisecond level is achieved through digital signal processing. For real-time processing scenarios, a dynamic adjustment mechanism is used to continuously optimize the delay based on the real-time changes in the audio stream.
[0007] Preferably, in step S2, the phase-aligned channel signal is decomposed into a time-frequency matrix using an improved short-time Fourier transform. Each matrix contains a time index, a frequency index, and a complex spectrum value, forming a three-dimensional data structure. Based on the time-frequency matrix, the dominant frequency sequence is extracted within each time frame. Energy-weighted filtering is then used to suppress harmonic components, generating a fundamental frequency volume. This process is as follows: In the formula, This represents the fundamental frequency of the current time frame t. Indicates the length of the time frame. Indicates autocorrelation delay, These represent the minimum and maximum delay thresholds, respectively, and M represents the spectrum length. The energy weighting function is expressed as: , Indicates the attenuation coefficient. This indicates the estimated fundamental frequency; Using the fundamental frequency matrix as a reference, linear predictive coding analysis is applied to the time-frequency matrix to locate formants, as follows: In the formula, Represents time t. The energy of the m-th resonance peak. Indicates the spectral amplitude. The standard deviation of local noise at frequency f. This represents the bandwidth control parameter for the m-th resonance peak. This indicates the search frequency band based on the fundamental frequency harmonics, and finally the formant matrix is constructed.
[0008] Preferably, in step S2, the energy change rate of each unit is calculated through time-frequency gradient analysis of the time-frequency unit. and phase mutation rate By setting a dynamic threshold to identify energy mutation points, and using wavelet packet decomposition to finely separate the mutation regions, the following can be achieved: In the formula, The transient flag represents time t, where 1 indicates a transient event and 0 indicates a non-transient event. These represent the energy and phase change thresholds, respectively. These represent the energy and the standard deviation of the phase change in the current frame, respectively.
[0009] Preferably, in step S3, a three-dimensional Cartesian coordinate system is established, where the x-axis represents the positions of the left and right sound sources, the y-axis represents the positions of the front and rear sound sources, and the z-axis represents the frequency height; the fundamental frequency of the current frame t is extracted from the fundamental frequency volume. Calculate normalized height coordinates: Extract the first resonance peak from the output resonance matrix. The horizontal coordinate x is calculated as follows: ; Extracting the second resonance peak The depth coordinate y is calculated as follows: : Check the coordinate changes of adjacent frames, retain only the coordinates of the physically audible range, and generate three-dimensional spatial coordinates.
[0010] Preferably, in step S4, based on the three-dimensional spatial coordinates output by S3, the space is divided into 27 physically meaningful sub-regions, each corresponding to specific acoustic characteristics. For each time frame, the instantaneous energy of each sub-region is statistically analyzed. In the formula, This represents the total energy of the sub-region in the current time frame t. Represents the instantaneous energy of a time-frequency unit; Calculate the global target energy: Dynamically adjust the benchmark for each region. A dynamic energy gradient field is constructed to ensure the conservation of total energy after adjustment, as follows: In the formula, This represents the energy deviation of region t frames. Preferably, in step S4, the dynamic gain coefficient is calculated for each region, implemented as follows: In the formula, This represents the dynamic gain coefficient for region t. Indicates the energy flow coefficient. This represents the total global energy.
[0011] Preferably, step S4 is implemented by performing time smoothing through a first-order low-pass filter, as follows: In the formula, This represents the gain coefficient after smoothing the current frame. Indicates the smoothing coefficient; Preferably, step S4 is implemented by performing time smoothing through a first-order low-pass filter, as follows: In the formula, This represents the gain coefficient after smoothing the current frame. Indicates the smoothing coefficient; Calculate the total energy after adjustment. Implementation as follows: .
[0012] Preferably, in step S5, an impulse response library containing 27 predefined acoustic scenes is constructed. Each scene corresponds to 27 sub-regions divided by coordinates in step S3. Each impulse response includes: acoustic environment features, spatial acoustic parameters, and matching weights. Based on the output region and energy deviation value, the closest preset value candidate impulse responses are selected from the database. The best match is determined by the Euclidean distance between the coordinates and the preset coordinates of the impulse response. The scene with the closest depth coordinates is matched first. The output signal after gain adjustment is converted into a time-domain digital signal. The signal is processed by frame segmentation. A fast convolution operation based on FFT is performed on each frame of the signal. The FFT size is dynamically adjusted according to the audio content. The convolution result is corrected and verified in the frequency domain to generate the final audio signal.
[0013] Preferably, in step S6, key data of the current time frame are extracted synchronously from S3, S4, and S5; points in S3 whose coordinates are out of range are filtered; outliers in S5 whose spatial consistency scores are lower than a preset threshold are repaired; and the regional energy deviation of S4 and the matching weight of S5 are merged to generate a dynamic weight matrix. Based on the coordinate sequence of S3 and the spatial consistency score of S5, spatial dimensions are extracted, and the energy deviation of S4 and the convolution result of S5 are fused to generate: regional energy balance, dynamic range, and low-frequency compensation intensity. The optimization focus is automatically switched according to the audio content type, and spatial positioning parameters and energy balance parameters are generated first. The parameter weights are dynamically adjusted through the real-time audio analysis module, and a first-order low-pass filter is applied to the generated parameters to ensure that the changes in sound field width, reverberation time, etc. are gradual, generating a mixing parameter package containing four dimensions.
[0014] Preferably, a multi-channel mixing processing system includes a dynamic phase alignment delay calibration module, a multi-dimensional frequency decomposition and separation module, a three-dimensional spatial coordinate mapping module, an energy conservation field gain adjustment module, an intelligent impulse response dynamic convolution module, a multi-dimensional mixing parameter generation module, and a rendering output engine module.
[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention achieves high-precision separation of fundamental frequency volume, formant matrix, and transient profile through improved short-time Fourier transform and linear predictive coding analysis, and multi-dimensional frequency deconstruction technology. The extraction of the fundamental frequency volume can enhance the fundamental frequency energy, improve speech clarity or the fullness of instrument timbre; the accurate positioning of the formant matrix can simulate the spatial reflection characteristics of different instruments; and wavelet packet decomposition captures transient events while preserving the impact of the sound. The adaptive window length design combined with energy-weighted filtering effectively reduces computational complexity and is compatible with multiple scenario requirements. The three types of spectral components after separation can independently adjust gain or delay, significantly improving the controllability of audio processing. 2. This invention maps time-frequency units to a three-dimensional Cartesian coordinate system, achieving a direct correlation between sound source location and auditory perception. After coordinate transformation, the spatial distribution information of the sound source can drive the rendering engine, allowing listeners to perceive the dynamic movement trajectory of the sound source. By checking the physical audible range and filtering out abnormal coordinates, the rationality of the spatial mapping is ensured. Smoothing of adjacent frame coordinates further eliminates the sense of sound source jumps, making the trajectory of the moving sound source continuous and natural, significantly enhancing the realism of dynamic scenes. Ultimately, the three-dimensional coordinate transformation not only improves the immersiveness of spatial audio but also provides accurate sound source localization for multi-channel mixing; 3. This invention employs a dynamic gain adjustment mechanism divided into 27 sub-regions. Through adaptive correlation between depth coordinates and energy distribution, and dynamic energy gradient field design, it ensures that the total energy after adjustment is consistent with the original signal, avoiding volume fluctuations caused by gain adjustment. Especially in musical climaxes or movie explosion scenes, it can highlight key sound sources without disrupting the overall balance due to local overload. The smoothing processing of the first-order low-pass filter makes the gain change transition naturally, avoiding abrupt auditory changes, while meeting the requirements of real-time audio processing. Finally, it achieves fine control of energy distribution in multi-channel mixing, significantly improving the dynamic range of audio. Attached Figure Description
[0016] Figure 1 The following is the operation flow of a multi-channel mixing processing method according to the present invention. Figure 1 ; Figure 2 The following is the operation flow of a multi-channel mixing processing method according to the present invention. Figure 2 ; Figure 3 The following is the operation flow of a multi-channel mixing processing method according to the present invention. Figure 3 ; Figure 4 The following is the operation flow of a multi-channel mixing processing method according to the present invention. Figure 4 ; Figure 5 The following is the operation flow of a multi-channel mixing processing method according to the present invention. Figure 5 ; Figure 6This is a schematic diagram of the structure of a multi-channel mixing processing system according to the present invention. Detailed Implementation
[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example
[0018] Please see Figures 1-6 As shown, the present invention provides a technical solution comprising the following steps: S1. Based on the original multi-channel audio, perform millisecond-level delay calibration between channels through dynamic phase alignment; S2. Based on the phase-aligned channel signal, the fundamental frequency volume, formant matrix, and transient profile are separated using a multi-dimensional frequency deconstruction method. S3. Based on the separated spectral components, perform the transformation from time-frequency units to three-dimensional spatial coordinates; S4. Based on the three-dimensional spatial coordinate distribution, perform piecewise dynamic gain adjustment of the energy conservation field; S5. The signal after gain adjustment is dynamically convolved through an intelligently matched impulse response database. S6. Integrate all preceding processed data to generate multi-dimensional mixing parameters in real time.
[0019] In this embodiment, step S1 initiates multi-channel audio capture via the audio acquisition API to capture raw audio data. The captured raw data undergoes preliminary preprocessing, including normalization, baseline correction, and elimination of initial noise interference.
[0020] Specifically, a channel is selected as a reference channel, and the phase difference between the other channels and the reference channel is calculated through cross-correlation analysis, which is achieved as follows: In the formula, This represents the weighted total phase difference between the target channel and the reference channel. This represents the instantaneous energy of the i-th frequency component. This represents the phase angle of the reference channel at the i-th frequency component. Let N represent the phase angle of the target channel at the i-th frequency component, and N represent the total number of frequency components. The phase difference is converted into a time delay, implemented as follows: f represents frequency.
[0021] Based on the phase difference calculation results, the amount of delay that needs to be adjusted for each channel is determined. Precise delay at the millisecond level is achieved through digital signal processing. For real-time processing scenarios, a dynamic adjustment mechanism is used to continuously optimize the delay based on the real-time changes in the audio stream.
[0022] In this embodiment, step S2 performs time-frequency decomposition on the phase-aligned channel signal using an improved short-time Fourier transform, decomposing the input signal into a time-frequency unit matrix. Each unit contains a time index, a frequency index, and a complex spectrum value, forming a three-dimensional data structure.
[0023] Specifically, based on the time-frequency matrix, the dominant frequency sequence is extracted within each time frame. Energy-weighted filtering is then used to suppress harmonic components, generating the fundamental frequency volume, as follows: In the formula, This represents the fundamental frequency of the current time frame t. Indicates the length of the time frame. Indicates autocorrelation delay, These represent the minimum and maximum delay thresholds, respectively, and M represents the spectrum length. The energy weighting function is expressed as: , Indicates the attenuation coefficient. This indicates the estimated fundamental frequency; In this embodiment, using the fundamental frequency matrix as a reference, linear predictive coding analysis is applied to the time-frequency matrix to locate the formant peaks, as follows: In the formula, Represents time t. The energy of the m-th resonance peak. Indicates the spectral amplitude. The standard deviation of local noise at frequency f. This represents the bandwidth control parameter for the m-th resonance peak. This indicates the search frequency band based on the fundamental frequency harmonics, and finally the formant matrix is constructed.
[0024] In this embodiment, step S2 calculates the energy change rate of each unit through time-frequency gradient analysis of the time-frequency unit. and phase mutation rate By setting a dynamic threshold, identifying energy mutation points, and using wavelet packet decomposition to finely separate the mutation regions, the following can be achieved: In the formula, The transient flag represents time t, where 1 indicates a transient event and 0 indicates a non-transient event. These represent the energy and phase change thresholds, respectively. These represent the energy and the standard deviation of the phase change in the current frame, respectively.
[0025] In this embodiment, step S3 establishes a three-dimensional Cartesian coordinate system, where the x-axis represents the position of the left and right sound sources, the y-axis represents the position of the front and rear sound sources, and the z-axis represents the frequency height.
[0026] Specifically, the fundamental frequency of the current frame t is extracted from the fundamental frequency volume. Calculate the normalized height coordinates: Extract the first resonance peak from the output resonance matrix. The horizontal coordinate x is calculated as follows: Extracting the second resonance peak Calculate the depth coordinate y, which is achieved as follows:
[0027] Examine the coordinate changes between adjacent frames, retain only the coordinates of the physically audible range, and generate three-dimensional spatial coordinates.
[0028] In this embodiment, S4 divides the space into 27 sub-regions with clear physical meaning based on the three-dimensional spatial coordinates output by S3, with each region corresponding to specific acoustic characteristics.
[0029] Specifically, for each time frame, the instantaneous energy of each sub-region is calculated: In the formula, This represents the total energy of the sub-region in the current time frame t. Represents the instantaneous energy of a time-frequency unit; Calculate the global target energy: Dynamically adjust the benchmark for each region. A dynamic energy gradient field is constructed to ensure the conservation of total energy after adjustment, as follows: In the formula, , representing the energy deviation of region t frames. In this embodiment, step S4, calculating the dynamic gain coefficient for each region, is implemented as follows: In the formula, This represents the dynamic gain coefficient for region t. Indicates the energy flow coefficient. This represents the total global energy. In this embodiment, S4, through time smoothing processing using a first-order low-pass filter, is implemented as follows: In the formula, This represents the gain coefficient after smoothing the current frame. Indicates the smoothing coefficient; Calculate the total energy after adjustment. Implementation as follows: .
[0030] In this embodiment, step S5 involves constructing an impulse response library containing 27 predefined acoustic scenes. Each scene corresponds to 27 sub-regions divided by coordinates in step S3. Each impulse response includes: acoustic environment features, spatial acoustic parameters, and matching weights.
[0031] Specifically, based on the output region and energy deviation values, the closest preset value candidate impulse response is selected from the database. The best match is determined by the Euclidean distance between the coordinates and the preset coordinates of the impulse response. Priority is given to matching the scene with the closest depth coordinates. The output signal after gain adjustment is converted into a time-domain digital signal and the signal is processed by frame segmentation.
[0032] Fast convolution based on FFT performs convolution operation on each frame of signal, dynamically adjusts the FFT size according to the audio content, and performs frequency domain correction and verification on the convolution result to generate the final audio signal.
[0033] In this embodiment, step S6 synchronously extracts key data of the current time frame from S3, S4, and S5, filters out points in S3 whose coordinates are out of range, repairs outliers in S5 whose spatial consistency scores are lower than a preset threshold, and merges the regional energy deviation of S4 with the matching weight of S5 to generate a dynamic weight matrix.
[0034] Spatial dimensions are extracted based on the coordinate sequence of S3 and the spatial consistency score of S5. The energy bias of S4 and the convolution result of S5 are then fused to generate: regional energy balance, dynamic range, and low-frequency compensation intensity.
[0035] The system automatically switches optimization priorities based on the type of audio content, prioritizing the generation of spatial positioning parameters and energy balance parameters. The parameter weights are dynamically adjusted through a real-time audio analysis module, and a first-order low-pass filter is applied to the generated parameters to ensure smooth changes in sound field width, reverberation time, and other parameters, generating a mixing parameter package containing four dimensions.
[0036] In this embodiment, a multi-channel mixing processing system includes a dynamic phase alignment delay calibration module, a multi-dimensional frequency decomposition and separation module, a three-dimensional spatial coordinate mapping module, an energy conservation field gain adjustment module, an intelligent impulse response dynamic convolution module, a multi-dimensional mixing parameter generation module, and a rendering output engine module.
[0037] Working principle: The system captures raw audio data from multiple channels via an audio acquisition API, and performs preprocessing including normalization, baseline correction, and noise cancellation. Then, it selects one channel as a reference channel and uses cross-correlation analysis to calculate the phase difference between the other channels and the reference channel at each frequency component. Using a weighted total phase difference formula, combined with the instantaneous energy of the frequency components, the phase difference is converted into a time delay. Based on the calculation results, precise millisecond-level delay adjustments are applied to each channel, and a dynamic mechanism monitors changes in the audio stream in real time to continuously optimize the delay parameters. The phase-aligned channel signal is decomposed into a three-dimensional time-frequency unit matrix containing time, frequency, and complex spectral values using an improved short-time Fourier transform. Within each time frame, the fundamental frequency volume is extracted based on the dominant frequency sequence, and non-harmonic components are suppressed by energy-weighted filtering. Subsequently, using the fundamental frequency volume as a reference, linear predictive coding analysis is applied to identify formants, constructing a formant matrix containing formant energy, bandwidth control parameters, and a search frequency band. Simultaneously, energy and phase change rate are calculated through time-frequency gradient analysis, a dynamic threshold is set to identify transient events, and wavelet packet decomposition is used to finely separate abrupt change regions. Finally, three types of spectral components are output: the fundamental frequency volume, the formant matrix, and the transient profile. Using the frequency matrix and formant matrix as input, a three-dimensional Cartesian coordinate system is established. Normalized height coordinates of the current frame are extracted using the fundamental frequency, and the horizontal coordinate (x) is calculated using the first formant, while the depth coordinate (y) is calculated using the second formant. Subsequently, coordinate changes between adjacent frames are checked, filtering outomas exceeding the physically audible range to generate a three-dimensional spatial coordinate sequence consistent with auditory perception logic. Based on the S3 three-dimensional spatial coordinates, the space is divided into 27 physically meaningful sub-regions, each corresponding to specific acoustic characteristics. The instantaneous energy of each sub-region is statistically analyzed for each time frame, and the global target energy is calculated. By dynamically adjusting the reference energy and combining the energy flow coefficient and the global total energy, a regional dynamic gain coefficient is generated. Then, it should be... A first-order low-pass filter is used to smooth the gain coefficient over time, ensuring the conservation of total energy after adjustment while suppressing transient fluctuations, ultimately outputting a gain-adjusted signal that conforms to spatial distribution logic. An impulse response library containing 27 predefined acoustic scenes is constructed, each scene corresponding to a sub-region divided by S3. Based on the output region and energy deviation values, candidate impulse responses are screened, and the optimal matching scene is determined through Euclidean distance matching. The gain-adjusted signal is converted into a time-domain digital signal and processed frame by frame, with each frame processed using fast convolution based on FFT. The FFT size is dynamically adjusted according to the audio content, while the convolution results undergo frequency domain correction and spatial consistency verification to generate the final audio signal. The system extracts key data from the current time frame, filters out coordinate points outside the physical range in S3, and corrects outliers with spatial consistency scores below the threshold. A dynamic weight matrix is generated by merging the regional energy deviation from S4 with the matching weights from S5. Spatial dimensions are extracted based on the coordinate sequence and S5 spatial score, and the convolution results of S4 energy deviation and S5 are fused to generate parameters such as regional energy balance, dynamic range, and low-frequency compensation intensity. The optimization focus is automatically switched according to the audio content type, and parameter weights are dynamically adjusted through a real-time audio analysis module. A first-order low-pass filter is applied to ensure smooth changes in sound field width, reverberation time, etc., ultimately outputting mixing parameters encompassing four dimensions: spatial, energy, acoustic, and dynamic.
[0038] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their likenesses.
[0039] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.
Claims
1. A multi-channel mixing processing method, characterized by, The method comprises the following steps: S1. According to the multi-channel original audio, the millisecond-level delay calibration between channels is performed through dynamic phase alignment; S2. According to the phase-aligned channel signals, the separation of the fundamental frequency body, the formant matrix and the transient profile is performed through a multi-dimensional frequency decomposition method; S3. According to the separated spectral components, the conversion of the time-frequency unit to the three-dimensional space coordinates is performed; S4. According to the three-dimensional space coordinate distribution, the segmented dynamic gain adjustment of the energy conservation field is performed; S5. The gain-adjusted signal is subjected to dynamic convolution processing through an intelligent matching impulse response database; S6. The multi-dimensional mixing parameter real-time generation is performed by integrating all the processed data in the foregoing steps.
2. The multi-channel mixing processing method of claim 1, wherein: The S1 starts multi-channel audio capture through an audio collection API, captures original audio data, preliminarily preprocesses the captured original data, selects a channel as a reference channel, calculates phase differences between other channels and the reference channel through cross-correlation analysis, and realizes: In the formula, A weighted total phase difference between the target channel and the reference channel is represented by An instantaneous energy of the i-th frequency component is represented by A phase angle of the reference channel at the i-th frequency component is represented by A phase angle of the target channel at the i-th frequency component is represented by and N represents a total number of frequency components. The phase difference is converted into a time delay amount. According to the phase difference calculation result, the delay amount that needs to be adjusted for each channel is determined, and the millisecond-level accurate delay is performed through digital signal processing. For the real-time processing scene, the delay amount is continuously optimized according to the real-time change of the audio stream through a dynamic adjustment mechanism.
3. The multi-channel mixing processing method of claim 2, wherein: The S2, through the time-frequency decomposition of the phase-aligned channel signal by the improved short-time Fourier transform, decomposes the input signal into a time-frequency unit matrix, each unit containing a time index, a frequency index and a complex spectrum value, forming a three-dimensional data structure; based on the time-frequency matrix, extracts a main frequency sequence in each time frame, suppresses non-harmonic components through energy-weighted filtering, generates a basic frequency body, and realizes: , In the formula, denotes the fundamental frequency of the current time frame t, denotes the time frame length, denotes the autocorrelation delay, denotes the minimum delay threshold and the maximum delay threshold, respectively, and M denotes the spectrum length, denotes the energy weighting function; With the fundamental frequency body as the reference, the linear predictive coding analysis is applied to the time-frequency matrix to locate the formants, and the formula is: In the formula, t represents the time, and the energy of the mth formant, the spectral amplitude, the local noise standard deviation of the frequency f, the bandwidth control parameter of the mth formant, the search frequency band based on the fundamental frequency harmonic, and finally the formant matrix is constructed.
4. The multi-channel mixing processing method of claim 3, wherein: The S2, through the time-frequency gradient analysis of the time-frequency unit, calculates the energy change rate of each unit And phase mutation rate Set dynamic threshold, identify energy mutation point, fine separation of mutation area by wavelet packet decomposition, realize as: In the formula, Indicates the transient symbol of time t, Indicate energy and phase change threshold respectively, Indicate the energy and phase change standard deviation of the current frame respectively.
5. A multi-channel mixing method according to claim 4, characterized in that: S3, a three-dimensional Cartesian coordinate system is established, x-axis: representing the left and right sound source position, y-axis: representing the front and back sound source position, z-axis: representing the frequency height; extracting the basis frequency of the current frame t from the basis frequency body Calculate the normalized height coordinate: Extract the first formant of the output formant matrix Calculate the horizontal coordinate x, which is realized as: Extract the second formant Calculate the depth coordinate y, which is realized as: Check the coordinate change of adjacent frames, only keep the physically audible range coordinates, and generate three-dimensional space coordinates.
6. A multi-channel mixing method according to claim 5, wherein: The S4 divides the space into 27 sub-regions with clear physical meaning based on the output three-dimensional spatial coordinates, each region corresponds to a specific acoustic characteristic, and the instantaneous energy of each sub-region is counted for each time frame: In the formula, The total energy of the sub-region of the current time frame t is represented by The instantaneous energy of the time-frequency unit is represented by Calculate global target energy: ; Dynamic adjustment benchmark for each region ; Construct a dynamic energy gradient field to ensure total energy conservation after adjustment, which is: In the formula, Energy deviation of region t frame.
7. A multi-channel mixing processing method according to claim 6, characterized in that: S4. Computing the dynamic gain coefficient for each region, implemented as: In the formula, denotes the dynamic gain coefficient for region t, denotes the energy flow coefficient, denotes the global total energy.
8. The multi-channel mixing processing method of claim 7, wherein: The S4 is realized by time smoothing processing through a first-order low-pass filter, and is: In the formula, Gn represents the gain coefficient of the current frame after smoothing, Gn represents the smoothing coefficient; Adjusted total energy is calculated implemented as: .
9. The multi-channel mixing processing method of claim 1, wherein: The S5 comprises the following steps: constructing an impulse response library containing 27 predefined acoustic scenes, each scene corresponding to 27 sub-regions of the S3 coordinate division, each impulse response containing acoustic environment characteristics, spatial acoustic parameters and matching weights, selecting the closest preset value candidate impulse responses from the database based on the output region and energy deviation value, determining the best matching through the Euclidean distance between the coordinates and the preset coordinates of the impulse response, preferentially matching the scene closest to the depth coordinates, converting the gain-adjusted signal into a time-domain digital signal, performing frame processing on the signal, performing convolution operation on each frame signal based on the fast convolution of FFT, dynamically adjusting the FFT size according to the audio content, and performing frequency domain correction and verification on the convolution result to generate the final audio signal.
10. A multi-channel sound mixing processing system implemented by the multi-channel sound mixing processing method according to claim 1, wherein: The method comprises a dynamic phase alignment delay calibration module, a multi-dimensional frequency decomposition component separation module, a three-dimensional space coordinate mapping module, an energy conservation field gain adjustment module, an intelligent impulse response dynamic convolution module, a multi-dimensional mixing parameter generation module and a rendering output engine module.