Noise suppression and frequency response equalization dynamic optimization method of sound system in complex sound field environment

Abstract

This invention provides a dynamic optimization method for noise suppression and frequency response equalization of an audio system in complex sound field environments. The method includes acquiring spatial acoustic signals, auxiliary sensing signals, and system reference signals; constructing a feature fusion dataset based on a physical-acoustic coupling mapping mechanism; performing spatiotemporal modeling of the current sound field environment; and generating a sound field state vector and a sound field complexity index. Based on the multimodal feature fusion dataset and the sound field state vector, a frequency domain energy residual surrogate model is introduced as a physical embedded constraint to perform feature-level bidirectional joint solution for noise suppression and dynamic frequency response equalization, generating an initial filter coefficient set. This invention, through soft decision-making and convex set projection, ensures that the final output filter coefficients simultaneously satisfy auditory masking protection and physical frequency response compensation, effectively avoiding the conflict between equalization amplification of background noise and noise reduction creating spectral holes.

Description

Technical Field

[0001] This invention relates to the field of audio signal processing technology, specifically to a method for dynamic optimization of noise suppression and frequency response equalization of a sound system in a complex sound field environment. Background Technology

[0002] With the widespread application of intelligent speakers and sound reinforcement systems in complex acoustic environments such as large theaters, conference centers, and smart cockpits, extremely high demands are placed on the system's noise suppression and frequency response equalization capabilities under non-stationary noise, strong reverberation, and dynamic audience occupancy. Traditional audio system signal processing methods suffer from the following significant technical shortcomings:

[0003] Sound field perception is limited in scope and lacks physical priors and spatiotemporal dynamic modeling: Existing technologies typically rely solely on acoustic signals acquired by microphone arrays for noise estimation, neglecting the dynamic influence of environmental physical parameters (changes in sound velocity and absorption coefficient caused by temperature and humidity) and audience distribution (sound shadow zones and scattering distortion caused by human occlusion) on sound field boundary conditions; Purely data-driven sound field modeling lacks the constraints of physical laws, resulting in severe lag and distortion in feature extraction in scenarios with strong reflections or sudden occlusion.

[0004] Noise suppression and frequency response equalization are processed independently, resulting in spectral conflicts and mutual losses: Traditional solutions usually process noise reduction and equalization as cascaded independent modules. When suppressing noise, the noise reduction module often over-compresses the spectrum, causing spectral holes in the subsequent equalization module. When the equalization module boosts certain frequency bands, it amplifies the masked background noise. The two lack a joint solution mechanism, which can easily lead to perceptual distortions such as sound coloration and loss of musicality.

[0005] The system response is lagging and lacks forward-looking prediction and dynamic adaptation capabilities: Most existing systems are passive lag mechanisms of perception-response. When the sound field changes abruptly (such as the audience suddenly moving or a sudden burst of strong noise), the update of the filter coefficients always lags behind the change in the sound field, causing obvious auditory abnormalities (such as trailing and popping sounds).

[0006] The fixed edge model is difficult to adapt to long-tail scenarios and is prone to catastrophic forgetting: the parameters of the deployed audio system model are usually fixed and cannot be continuously optimized for specific environments. Even if some systems introduce online learning, they often destroy the existing model weights when adapting to new sound fields, resulting in catastrophic forgetting. Furthermore, it is impossible to use historical conflict experience to adjust the resolution of the underlying physical perception. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a dynamic optimization method for noise suppression and frequency response equalization of audio systems in complex sound field environments, thereby solving the problems mentioned in the background section.

[0008] To achieve the above objectives, the present invention provides a method for dynamic optimization of noise suppression and frequency response equalization of an audio system in a complex sound field environment, comprising the following steps:

[0009] S1: Acquire spatial acoustic signals, auxiliary sensing signals and system reference signals, and construct a feature fusion dataset based on the physical-acoustic coupling mapping mechanism to perform spatiotemporal modeling of the current sound field environment and generate sound field state vectors and sound field complexity indices;

[0010] S2: Based on the multimodal feature fusion dataset and sound field state vector, a frequency domain energy residual surrogate model is introduced as a physical embedded constraint to perform feature-level bidirectional joint solution of noise suppression and dynamic frequency response equalization, generating an initial filter coefficient set;

[0011] S3: Based on the sound field state vector and the short-time sound field state prediction network, perform feedforward correction on the initial filter coefficient group to obtain updated filter coefficients. Based on the sound field complexity index, dynamically and adaptively adjust the network computing power and physical solution accuracy. Smoothly distribute the updated filter coefficients and perform iterative optimization. At the same time, use the spectral conflict divergence index generated by incremental learning as the basis for adjusting the physical prior resolution and feed it back to step S1.

[0012] As a preferred embodiment, the construction of the feature fusion dataset based on the physical-acoustic coupling mapping mechanism in S1 includes:

[0013] The system acquires ambient sound pressure fluctuations and direct reflection aliasing information captured by the microphone array as spatial acoustic signals, acquires temperature and humidity data and millimeter-wave radar data as auxiliary signals, and acquires the original audio electrical signal to be played from the audio terminal as the system reference signal.

[0014] The listener distribution point cloud acquired by radar is transformed into a time-varying acoustic impedance scatterer distribution map based on the standard acoustic impedance coefficient and volume ratio of human tissue. Combined with the sound velocity model corrected by temperature and humidity data, a time-varying acoustic boundary condition tensor is constructed.

[0015] The time-varying acoustic boundary condition tensor is transformed into a geometric topological attention bias matrix. Injected into the CNN-Transformer hybrid architecture, the weight matrix of the multi-head self-attention mechanism is physically constrained and redistributed, with the specific formula as follows: This forces the network to focus on the sound shadow area and scattering distortion caused by human occlusion or movement, and output geometric topology sensing features.

[0016] in, This represents the geometric topology-aware features that incorporate physical space constraints. Let represent the activation function, Q represent the query matrix, K represent the key matrix, d represent the dimension of the key vector, and V represent the value matrix of the acoustic feature information that actually needs to be transmitted.

[0017] As a preferred embodiment, the method for generating the sound field complexity index includes:

[0018] The covariance matrix of the microphone array is decomposed into eigenvalues, and the ratio of the largest eigenvalue to the trace is extracted as the spatial coherence of the sound field.

[0019] And calculate the inter-frame difference of the spatial coherence of the sound field as the non-stationary transient rate;

[0020] The spatial coherence of the sound field and the non-stationary transient rate are fused to generate a quantified sound field complexity index.

[0021] As a preferred embodiment, the characteristic-level bidirectional joint solution for noise suppression and dynamic frequency response equalization in step S2, which generates the initial filter coefficient set, includes:

[0022] The acoustic field state vector and the system reference signal are input into a physically constrained adaptive filter for echo cancellation to obtain the residual signal.

[0023] The residual signal and the sound field state vector are input into a lightweight convolutional recurrent network. The frequency domain physical constraint mechanism and the space-psychoacoustic joint masking surface are combined to perform dual physical embedded constraint extraction processing to obtain the perceptual weighted noise reduction gain flow.

[0024] Dynamic target frequency response approximation processing is performed based on the broadband room impulse response and background noise floor measured online to obtain the frequency response compensation coefficient flow;

[0025] The perceptual weighted noise reduction gain stream and the frequency response compensation coefficient stream are subjected to spectral conflict decoupling and alternating iterative projection processing using an alternating projection joint solution module based on spectral masking and compensation constraints to obtain the initial filter coefficient set.

[0026] As a preferred embodiment, the dual physical embedded constraint extraction process includes:

[0027] The time-varying acoustic boundary condition tensor generated by S1 is used as the frequency domain energy conservation constraint correction term for the room normal mode, and the energy residual of the frequency domain acoustic transfer function is calculated using the frequency domain energy residual surrogate model as the regularization term of the loss function.

[0028] The masking threshold calculated based on the psychoacoustic model is combined with the three-dimensional coordinates of the audience obtained by millimeter-wave radar to calculate the spatial spectral distortion compensation of the sound field, and a spatial-psychoacoustic joint masking surface is constructed. This surface is then transformed into an adaptive amplitude truncation boundary of the activation function.

[0029] As a preferred embodiment, the step of using the alternating projection joint solution module based on spectral masking and compensation constraints to perform spectral conflict decoupling and alternating iterative projection processing on the perceptual weighted noise reduction gain stream and frequency response compensation coefficient stream includes:

[0030] A spectrum conflict decoupling gating mechanism is introduced to extract the output gain conflict metric of the weighted noise reduction gain stream and the frequency response compensation coefficient stream in the same frequency band. The cosine similarity of the rate of change of the spectrum envelope of the two streams is calculated. When the similarity is less than 0, it is determined to be a conflict frequency band.

[0031] Introducing learnable gating scalars The weighted noise reduction gain stream will be output. With frequency response compensation coefficient current output Adaptive interpolation fusion is performed in conflicting frequency bands: ,in This indicates that the shared multilayer perceptron dynamically outputs data based on the current frequency band masking margin and frequency response deficiency.

[0032] fusion gain Projecting onto the psychoacoustic masking constraint set, i.e., projecting onto the set. Then project it onto the frequency response compensation constraint set. The two-step projection is executed alternately until the preset convergence condition is met;

[0033] in, This represents the set of filter gains that satisfy psychoacoustic masking constraints. This represents the gain vector of the filter. This represents the gain value in frequency band f. This represents the masking threshold on frequency band f. Let H represent the set of filter gains that satisfy the frequency response compensation constraint, and let H represent the transfer function of the environment. This refers to the frequency response actually heard. This represents the frequency response curve of a dynamic target. Denotes the square of the L2 norm. This indicates the error threshold.

[0034] In a preferred embodiment, the step of performing feedforward correction on the initial filter coefficient set based on the sound field state vector and the short-time sound field state prediction network includes:

[0035] The short-time sound field state prediction network is used as the predictor of the model prediction control. Based on the sound field state vectors of the current frame and historical frames, it predicts the trend of sound field state change in the future preset time delay window.

[0036] In the prediction time domain, the future control sequence is solved and the first element is taken as the feedforward correction amount. The noise reduction and equalization coefficients are calculated and updated smoothly in advance.

[0037] The predicted future sound field state vector is fed forward into the joint cost function supernetwork, and the cost function weights are pre-adjusted in advance based on the predicted situation.

[0038] As a preferred embodiment, the construction and updating method of the joint cost function hypernetwork includes:

[0039] The current frame sound field state vector, the future sound field state vector predicted by the short-time sound field state prediction network, the physical constraint residual of the frequency domain energy residual surrogate model, and the spectral conflict divergence index jointly solved by alternating projection are concatenated and input into a weight generator consisting of three fully connected layers. The output is a normalized […]. , , , Weight vector;

[0040] Constructed joint cost function: ;

[0041] in, Indicates the combined value of the system. This represents the weighting coefficient for the mean squared error loss. This represents the mean square error between the system output signal and the target pure signal. This represents the weighting coefficient for the perceived spectral distortion penalty. This represents the perceptual spectral distortion penalty term based on the Buck scale. This represents the weighting coefficient for the spectral flatness penalty. This represents the spectral fluctuation penalty term. This represents the weighting coefficient for hardware dynamic margin constraints. This indicates the power amplifier's thermal protection and clipping distortion limitations;

[0042] Increase in strong non-stationary disturbance scenarios Increase in deep reverberation standing wave scenarios Increase in advance in dynamic mutation scenarios Dynamic equilibrium occurs when the spectral conflict divergence index exceeds the limit. and The proportions.

[0043] As a preferred embodiment, the dynamic adaptive allocation of network computing power and physics solution accuracy based on the sound field complexity index includes:

[0044] A first threshold and a second threshold are preset, and the first threshold is less than the second threshold;

[0045] When the sound field complexity index is below the first threshold, the sound field is determined to be stable. Dynamic channel pruning is performed on the convolutional recurrent network to a preset low power ratio, and a low-order IIR filter or a low-tap FIR filter is used.

[0046] When the complexity index exceeds the second threshold, a sudden change in the sound field is detected, all network channels are activated, and the system switches to a higher-order FIR filter.

[0047] The number of spatial grid computation branches activated and the number of alternating projection iterations are dynamically adjusted in step S2 of the frequency domain energy residual proxy model solution based on the computing power budget.

[0048] In a preferred embodiment, the updated filter coefficients are smoothly distributed and iteratively optimized, while the spectral conflict divergence index generated by incremental learning is used as the basis for adjusting the physical prior resolution and fed back to step S1, including:

[0049] The updated coefficients are smoothly loaded into the digital signal processor using sinusoidal interpolation, and the frequency response deviation and residual noise power error of the output signal are monitored.

[0050] During the device's idle period, edge incremental learning is performed on a lightweight network, including a front-end feature extraction network and a frequency domain energy residual proxy model, using error data, buffered spectral conflict divergence index, and corresponding acoustic field features.

[0051] An elastic weight consolidation mechanism is introduced, the Fisher information matrix of network parameters with respect to the historical sound field dataset is calculated, and a parameter importance penalty term is added to the fine-tuning loss function;

[0052] Based on the accumulated conflict divergence index, the activation strategy of the spatial grid calculation branch of the physical constraint layer of the mid-frequency domain energy residual surrogate model in S2 is adaptively adjusted, and the spectral conflict divergence index is fed back to S1 as the basis for adjusting the spatial resolution weight when the radar point cloud is converted into an impedance scatterer distribution map.

[0053] This invention provides a method for dynamic optimization of noise suppression and frequency response equalization of an audio system in a complex sound field environment, which has the following beneficial effects:

[0054] This invention introduces millimeter-wave radar point cloud and temperature and humidity data, transforms them into time-varying acoustic impedance scatterer distribution maps and sound velocity correction models, and constructs a physical-acoustic coupling mapping mechanism. It transforms physical boundary conditions into the geometric topological attention bias matrix of Transformer, forcing the network to focus on the sound shadow region and scattering distortion, thus overcoming the deficiency of pure data-driven models lacking physical priors. At the same time, it uses MPC to predict the state vector back to perform Kalman gain correction, enabling the perception module to align with the direction of sudden changes in the sound field in advance, completely eliminating the lag in feature extraction.

[0055] On the one hand, this invention introduces a frequency domain energy residual proxy model and a spatial-psychoacoustic joint masking surface to form dual physical embedded constraints in the energy domain and the perception domain at the feature extraction layer, ensuring that the network output follows the acoustic energy conservation and human ear masking effect. On the other hand, it innovatively designs a spectral conflict decoupling gating and alternating iterative projection mechanism. Through soft decision-making and convex set projection, the filter coefficients of the final output simultaneously satisfy auditory masking protection and frequency response physical compensation, effectively avoiding the conflict between equalizing amplified background noise and creating spectral holes for noise reduction.

[0056] This invention utilizes a short-time sound field state prediction network combined with model predictive control to predict sound field boundary abrupt changes and standing wave evolution 50ms-100ms in advance, calculate feedforward corrections in advance and smoothly update coefficients, achieving a leap from passive response to active look-ahead. Furthermore, the constructed joint cost supernetwork can dynamically generate cost function weights end-to-end based on the current sound field state, predicted situation, and spectral conflict divergence. It prioritizes maintaining the signal-to-noise ratio under strong noise, severely punishes spectral fluctuations under deep reverberation, and prevents clipping distortion before abrupt changes, thus achieving adaptive evolution of the system optimization objective with the physical situation.

[0057] This invention, based on a quantized sound field complexity index, achieves stepless smooth allocation of computing power: dynamic pruning and order reduction filtering are used to save power during stable periods, while full network and high-order FIR are activated during abrupt changes to maintain accuracy. This completely solves the resource waste or performance bottleneck caused by fixed computing power. More importantly, it constructs a cross-cycle closed-loop feedback mechanism from spectral conflict divergence to radar point cloud spatial resolution weights, and uses an elastic weight consolidation mechanism to perform edge incremental learning during device idle periods. This not only prevents catastrophic forgetting but also allows the physical prior resolution to be automatically refined based on historical conflict experience, enabling the audio system to have the ability to continuously self-optimize in complex and ever-changing environments. Attached Figure Description

[0058] Figure 1 This is a flowchart of the dynamic optimization method for noise suppression and frequency response equalization of a sound system in a complex sound field environment according to the present invention.

[0059] Figure 2 This is a flowchart of the dynamic optimization method for noise suppression and frequency response equalization of a sound system in a complex sound field environment according to the present invention.

[0060] Figure 3 This is a flowchart illustrating the process of obtaining the initial filter coefficient set in this embodiment of the present invention.

[0061] Figure 4 This is a flowchart illustrating the initial filter coefficient update process of this embodiment of the present invention;

[0062] Figure 5 This is a flowchart of the predictive control and iterative optimization process of the present invention. Detailed Implementation

[0063] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0064] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides a method for dynamic optimization of noise suppression and frequency response equalization of an audio system in a complex sound field environment, including the following steps:

[0065] S1: Acquire spatial acoustic signals, auxiliary sensing signals and system reference signals, and construct a feature fusion dataset based on the physical-acoustic coupling mapping mechanism to perform spatiotemporal modeling of the current sound field environment and generate sound field state vectors and sound field complexity indices;

[0066] In this embodiment, the spatial acoustic signal is specifically the ambient sound pressure fluctuation and direct reflection aliasing information captured by the microphone array; the auxiliary sensing signal is specifically the temperature and humidity and the physical environment and audience occupancy status obtained by the millimeter-wave radar; the system reference signal is specifically the original audio electrical signal to be played by the audio terminal; and the sound field state vector is specifically an encoded vector including high-dimensional sound field characteristic parameters such as noise type, signal-to-noise ratio, reverberation time and direct reverberation energy ratio.

[0067] Specifically, the array manifold and covariance matrix are obtained through a multi-channel microphone array with a uniform circular topology, the air velocity and absorption coefficient are obtained through a temperature and humidity sensor, and the three-dimensional point cloud coordinates and distribution density of the audience are obtained through a millimeter-wave radar. The collected multi-source heterogeneous data are then time-stamped and formatted to form a multi-modal feature matrix with unified dimensions.

[0068] In this embodiment, the specific process for generating the sound field state vector and the sound field complexity index includes:

[0069] S11: Perform multimodal alignment and feature extraction processing on spatial acoustic signals, auxiliary sensing signals and system reference signals to obtain a multimodal feature matrix;

[0070] S12: The multimodal feature matrix is ​​subjected to time-varying boundary tensor construction and attention constraint processing using a physical-acoustic coupling mapping mechanism to obtain geometric topology-aware features;

[0071] Specifically, the processing flow of the multimodal physics-acoustic coupling mapping mechanism is as follows: the listener distribution point cloud acquired by radar is transformed into a time-varying acoustic impedance scatterer distribution map based on the standard acoustic impedance coefficient of human tissue and its volume ratio; a sound velocity model corrected by temperature and humidity data is combined to construct a time-varying acoustic boundary condition tensor; and this boundary tensor is transformed into a geometric topological attention bias matrix. Injected into the CNN-Transformer hybrid architecture, the weight matrix of the multi-head self-attention mechanism is physically constrained and redistributed, with the specific formula as follows:

[0072] This forces the network to focus on the sound shadow area and scattering distortion caused by human occlusion or movement, and output geometric topology sensing features.

[0073] in, This represents the geometric topology-aware features that incorporate physical space constraints. Let represent the activation function, Q represent the query matrix, K represent the key matrix, d represent the dimension of the key vector, and V represent the value matrix of the acoustic feature information that actually needs to be transmitted.

[0074] S13: Perform covariance matrix decomposition and transient rate calculation on the spatial acoustic signal to obtain the sound field complexity index;

[0075] Specifically, the covariance matrix of the microphone array is decomposed into eigenvalues, the ratio of the largest eigenvalue to the trace is extracted as the spatial coherence of the sound field, and its inter-frame difference is calculated as the non-stationary transient rate. The two are then fused to generate a quantized sound field complexity index.

[0076] S14: Perform fully connected encoding on the geometric topology sensing features and sound field complexity index to obtain the sound field state vector.

[0077] Specifically, geometric topology sensing features are concatenated with sound field complexity indicators, and a sound field state vector representing the dynamic changes of the current sound field is output through a fully connected layer encoding.

[0078] Furthermore, step S1 also includes two dynamic update mechanisms: First, the future sound field state prediction vector output by the model prediction control algorithm in subsequent step S3 is used as prior information to be fed back to the covariance matrix update module in this step. By introducing the predicted state covariance offset to perform Kalman gain correction, the perception module is aligned with the potential direction of sound field change in advance, eliminating the lag in feature extraction. Second, the spectral conflict divergence index generated by incremental learning in subsequent step S3 is used to adaptively adjust the spatial resolution weight when the radar point cloud is converted into an impedance scatterer distribution map. When historical conflicts are severe, the grid calculation branch of the listener scatterer edge of the physical perception layer is automatically refined and activated and the activation state of the branch is fixed, realizing data-driven cross-cycle update of physical prior resolution.

[0079] S2: Based on the multimodal feature fusion dataset and sound field state vector, a frequency domain energy residual surrogate model is introduced as a physical embedded constraint to perform feature-level bidirectional joint solution of noise suppression and dynamic frequency response equalization, generating an initial filter coefficient set;

[0080] In this embodiment, based on the multimodal feature fusion dataset and the sound field state vector, a frequency domain energy residual surrogate model is introduced as a physical embedded constraint. Feature-level bidirectional joint solution for noise suppression and dynamic frequency response equalization is performed to generate an initial filter coefficient set, including the following steps:

[0081] S21: Input the sound field state vector and the system reference signal into the physical constraint adaptive filter for echo cancellation to obtain the residual signal;

[0082] Specifically, the sound field state vector and the system reference signal are input into a physical constraint adaptive filter based on fast recursive least squares, and fast linear echo cancellation is performed to obtain the residual signal.

[0083] S22: Input the residual signal and the sound field state vector into a lightweight convolutional recurrent network, and perform dual physical embedded constraint extraction processing by combining the frequency domain physical constraint mechanism and the spatial-psychoacoustic joint masking surface to obtain the perceptual weighted noise reduction gain stream.

[0084] Specifically, a frequency domain physical constraint mechanism is introduced into the network output feature layer: the time-varying acoustic boundary condition tensor generated by S1 is used as the frequency domain energy conservation constraint correction term of the room normal mode. The energy residual of the frequency domain acoustic transfer function is calculated using a frequency domain energy residual surrogate model as the regularization term of the loss function, forcing the network feature extraction to follow the law of conservation of acoustic propagation energy under the current physical boundary. At the same time, the masking threshold calculated based on the psychoacoustic model is combined with the three-dimensional coordinates of the audience obtained by the millimeter-wave radar in S1 to calculate the spatial spectrum distortion compensation of the sound field based on the pre-calculated RTF (room transfer function) spatial interpolation lookup table or lightweight approximation model, and a spatial-psychoacoustic joint masking surface is constructed. This surface is transformed into the adaptive amplitude truncation boundary of the activation function, forming dual physical embedded constraints in the energy domain and the perception domain, respectively, and generating a perception-weighted noise reduction gain flow (hereinafter referred to as noise reduction flow).

[0085] S23: Based on the online measured broadband room impulse response and background noise floor, dynamic target frequency response approximation processing is performed to obtain the frequency response compensation coefficient flow;

[0086] Specifically, based on the broadband room impulse response measured online, the minimum phase approximation inverse filter is solved in the frequency domain. The fixed flat frequency response target is abandoned. The desired loudness of each frequency band is dynamically adjusted according to the masking effect of background noise and the equal loudness curve to match the subjective hearing of the human ear. The variable order frequency response compensation coefficient flow (hereinafter referred to as equalization flow) is calculated.

[0087] S24: The perceptual weighted noise reduction gain stream and the frequency response compensation coefficient stream are subjected to spectral conflict decoupling and alternating iterative projection processing using the alternating projection joint solution module based on spectral masking and compensation constraints to obtain the initial filter coefficient group.

[0088] Specifically, such as Figure 3 As shown, a spectrum conflict decoupling gating mechanism is introduced: the denoising stream uses the masking threshold feature as the attention layer of the equalization stream for both Key and Value inputs to prevent the equalization boost from amplifying the masked background noise; the equalization stream uses the frequency response correction amplitude feature as the attention layer of the denoising stream for both Key and Value inputs to prevent excessive compression of the equalization spectrum from causing holes in the equalization spectrum; both streams use a learnable gating threshold to softly decide gradient conflicts, providing initial values ​​for soft fusion of conflicting frequency bands, accelerating the convergence speed of subsequent convex set projection iterations, avoiding projection oscillations, and eliminating the conflicts between the two streams through an alternating iterative projection algorithm, outputting the initial filter coefficient set.

[0089] The specific process of the soft decision includes: extracting the output gain conflict metric of the noise reduction stream and the equalization stream in the same frequency band, calculating the cosine similarity of the rate of change of the spectral envelope of the two streams, and determining the conflict frequency band when the similarity is less than 0.

[0090] Introducing learnable gating scalars , output noise-reduced stream With balanced flow output Adaptive interpolation fusion is performed in conflicting frequency bands: ,

[0091] in This indicates that the shared multilayer perceptron dynamically outputs the signal based on the current frequency band masking margin and frequency response deficiency, and then uses the Sigmoid activation function to... Mapping to the [0, 1] interval to achieve soft decision-making;

[0092] Alternating iterative projection includes: merging the gain Projecting onto the psychoacoustic masking constraint set, i.e., projecting onto the set. The forced constraint gain must not be lower than the calculated masking threshold limit;

[0093] in, This represents the set of filter gains that satisfy psychoacoustic masking constraints. This represents the gain vector of the filter. This represents the gain value in frequency band f. This represents the masking threshold on frequency band f;

[0094] The projected gain is then projected onto the frequency response compensation constraint set, i.e., projected onto the set. The mean square error of the forced constraint gain on the target frequency response is minimized.

[0095] in, Let H represent the set of filter gains that satisfy the frequency response compensation constraint, and let H represent the transfer function of the environment. This refers to the frequency response actually heard. This represents the frequency response curve of a dynamic target. Denotes the square of the L2 norm. Indicates the error threshold;

[0096] The two-step projection is executed alternately until the preset convergence condition is met, so that the final output simultaneously meets the dual requirements of auditory masking protection and frequency response physical compensation.

[0097] Furthermore, during this process, the spectral conflict feature tensor output by the gating mechanism is extracted, the spectral conflict divergence index is calculated based on the feature tensor, and fed forward to S3 as the penalty basis for the rate of change of control quantity in the time series prediction network optimization and the driving input of the joint cost super network in S3.

[0098] S3: Based on the sound field state vector and the short-time sound field state prediction network, perform feedforward correction on the initial filter coefficient group to obtain updated filter coefficients. Based on the sound field complexity index, dynamically and adaptively adjust the network computing power and physical solution accuracy. Smoothly distribute the updated filter coefficients and perform iterative optimization. At the same time, use the spectral conflict divergence index generated by incremental learning as the basis for adjusting the physical prior resolution and feed it back to step S1.

[0099] In this embodiment, as Figure 4 As shown, based on the sound field state vector and the short-time sound field state prediction network, the initial filter coefficient set is subjected to feedforward correction to obtain updated filter coefficients. Based on the sound field complexity index, the network computing power and physics solution accuracy are dynamically and adaptively allocated. The updated filter coefficients are then smoothly distributed and iteratively optimized, including:

[0100] S31: The data-driven model predictive control algorithm based on the short-time sound field state prediction network is used to predict and prospectively adjust the sound field state vector to obtain the feedforward correction amount and the dynamic cost function weight.

[0101] Specifically, a short-time sound field state prediction network (such as LSTM / GRU) is used as the predictor for model predictive control. Based on the sound field state vectors of the current frame and historical frames, it predicts the sound field state change trend for the next 5 to 10 frames (corresponding to a time delay window of 50ms to 100ms) for slow-changing physical processes such as sound field boundary abrupt changes and low-frequency standing wave evolution. In the prediction time domain, the future control sequence is solved and the first element is taken as the feedforward correction amount to calculate and smoothly update the noise reduction and equalization coefficients in advance. At the same time, the predicted future sound field state vector is fed forward into the joint cost function supernetwork, and the joint cost function weights are dynamically generated in advance according to the sound field evolution trend. Based on the prediction trend, the cost function weights are pre-adjusted in advance. Furthermore, the sound field abrupt change trend predicted by data-driven MPC is used as a trigger signal to wake up the millimeter-wave radar in S1 in advance to increase the scanning frame rate, realizing predictive resource collaboration between the sensing end and the computing end.

[0102] S32: Construct a joint cost supernetwork, dynamically generate joint cost function weights based on the sound field state vector, prediction vector and spectral conflict divergence index, and update the initial filter coefficient set in combination with the feedforward correction amount;

[0103] Specifically, a joint cost supernetwork is constructed, which concatenates the current frame sound field state vector, the future sound field state vector predicted by the short-time sound field state prediction network, the physical constraint residual of the frequency domain energy residual surrogate model, and the spectral conflict divergence index jointly solved by alternating projections. This concatenation is then input into a weight generator consisting of three fully connected layers, and the output is normalized through the Softmax function. , , , Weight vector;

[0104] The mathematical expression for the constructed joint cost function is as follows:

[0105]

[0106] in, Indicates the combined value of the system. This represents the weighting coefficient for the mean squared error loss. This represents the mean square error between the system output signal and the target pure signal. This represents the weighting coefficient for the perceived spectral distortion penalty. This represents the perceptual spectral distortion penalty term based on the Buck scale. This represents the weighting coefficient for the spectral flatness penalty. This represents the spectral fluctuation penalty term. This represents the weighting coefficient for hardware dynamic margin constraints. This indicates the power amplifier's thermal protection and clipping distortion limitations;

[0107] It should be noted that in scenarios with strong non-stationary interference, the low signal-to-noise ratio and high non-stationary transient rate drive the increase in the supernetwork size. Values ​​are selected to prioritize minimizing signal error; in deep reverberation standing wave scenarios, long reverberation time states and high physical residuals drive the hypernetwork to increase The values ​​are chosen to strictly penalize color distortion caused by spectral fluctuations; in dynamic abrupt change scenarios, the deterioration trend of the sound field state predicted by the short-time sound field state prediction network drives the supernetwork to increase prematurely. The value is selected to prevent hardware clipping distortion caused by sudden changes in coefficients; when the spectral conflict divergence index exceeds the limit, the supernetwork dynamic equilibrium is reached. and The ratio of forced noise reduction and equalization is compromised at the cost level, and the cost function is made to evolve end-to-end adaptively with the physical characteristics of the sound field, the conflict state of the solution and the future evolution trend.

[0108] S33: Based on the sound field complexity index, the network computing power and physical solution accuracy are dynamically and adaptively adjusted to obtain the updated filter coefficients;

[0109] Specifically, such as Figure 5As shown, computing power is dynamically allocated according to the sound field complexity index quantized in S1: a first threshold and a second threshold are preset, with the first threshold being less than the second threshold; when the sound field complexity index is lower than the first threshold, the sound field is determined to be stable, and dynamic channel pruning is performed on the convolutional recurrent network to a preset low power consumption ratio, and a low-order IIR filter or a low-tap FIR filter is used to reduce computing power consumption and group delay; when the complexity index is higher than the second threshold, the sound field is determined to be abrupt, all network channels are awakened and switched to a high-order FIR filter to achieve accurate inverse filtering of low-frequency standing waves and spatial zero-point formation, and the computing power occupancy rate increases smoothly and steplessly with the complexity index; at the same time, the number of activated branches of the physical constraint layer spatial grid calculation and the number of alternating projection iteration rounds are dynamically adjusted according to the computing power budget when solving the frequency domain energy residual proxy model in step S2.

[0110] S34: Perform smooth loading and error monitoring on the updated filter coefficients, and use the error data to perform edge incremental learning to complete iterative optimization.

[0111] Specifically, the updated coefficients are smoothly loaded into the digital signal processor using sinusoidal interpolation, and the frequency response deviation and residual noise power error of the output signal are monitored.

[0112] Using error data and buffered spectral conflict divergence index and corresponding sound field features, edge incremental learning is performed on a lightweight network including a front-end feature extraction network and a frequency domain energy residual proxy model during device idle periods. The condition for determining the idle period is that the sound field complexity index is continuously lower than the first threshold for more than a preset time: the backbone layer of the network is frozen and only the fully connected layer is fine-tuned. An elastic weight consolidation (EWC) mechanism is introduced to calculate the Fisher information matrix of the network parameters with respect to the historical sound field dataset. A parameter importance penalty term is added to the fine-tuning loss function to prevent catastrophic forgetting when adapting to new sound fields.

[0113] The training data used for incremental learning comes from the spectral conflict divergence index and corresponding sound field features stored in the buffer during non-idle periods. During fine-tuning, the activation strategy of the spatial grid calculation branch of the physical constraint layer of the mid-frequency energy residual proxy model in S2 is adaptively adjusted based on the accumulated conflict divergence index. When historical conflicts are severe, the grid calculation branch of the listener scatterer edge is refined and activated and the activation state of the branch is fixed to achieve cross-cycle update. The spectral conflict divergence index is then fed back to S1 as the basis for adjusting the spatial resolution weight when the radar point cloud is converted into an impedance scatterer distribution map.

[0114] The present invention provides a method for dynamic optimization of noise suppression and frequency response equalization of audio systems in complex sound field environments.

[0115] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for dynamic optimization of noise suppression and frequency response equalization of an audio system in a complex sound field environment, characterized in that, Includes the following steps: S1: Acquire spatial acoustic signals, auxiliary sensing signals and system reference signals, and construct a feature fusion dataset based on the physical-acoustic coupling mapping mechanism to perform spatiotemporal modeling of the current sound field environment and generate sound field state vectors and sound field complexity indices; S2: Based on the multimodal feature fusion dataset and sound field state vector, a frequency domain energy residual surrogate model is introduced as a physical embedded constraint to perform feature-level bidirectional joint solution of noise suppression and dynamic frequency response equalization, generating an initial filter coefficient set; S3: Based on the sound field state vector and the short-time sound field state prediction network, perform feedforward correction on the initial filter coefficient group to obtain updated filter coefficients. Based on the sound field complexity index, dynamically and adaptively adjust the network computing power and physical solution accuracy. Smoothly distribute the updated filter coefficients and perform iterative optimization. At the same time, use the spectral conflict divergence index generated by incremental learning as the basis for adjusting the physical prior resolution and feed it back to step S1.

2. The method for dynamic optimization of noise suppression and frequency response equalization of a sound system in a complex sound field environment according to claim 1, characterized in that, The feature fusion dataset constructed in S1 based on the physical-acoustic coupling mapping mechanism includes: The system acquires ambient sound pressure fluctuations and direct reflection aliasing information captured by the microphone array as spatial acoustic signals, acquires temperature and humidity data and millimeter-wave radar data as auxiliary signals, and acquires the original audio electrical signal to be played from the audio terminal as the system reference signal. The listener distribution point cloud acquired by radar is transformed into a time-varying acoustic impedance scatterer distribution map based on the standard acoustic impedance coefficient and volume ratio of human tissue. Combined with the sound velocity model corrected by temperature and humidity data, a time-varying acoustic boundary condition tensor is constructed. The time-varying acoustic boundary condition tensor is transformed into a geometric topological attention bias matrix. Injected into the CNN-Transformer hybrid architecture, the weight matrix of the multi-head self-attention mechanism is physically constrained and redistributed, with the specific formula as follows: This forces the network to focus on the sound shadow area and scattering distortion caused by human occlusion or movement, and output geometric topology sensing features. in, This represents the geometric topology-aware features that incorporate physical space constraints. Let represent the activation function, Q represent the query matrix, K represent the key matrix, d represent the dimension of the key vector, and V represent the value matrix of the acoustic feature information that actually needs to be transmitted.

3. The method for dynamic optimization of noise suppression and frequency response equalization of a sound system in a complex sound field environment according to claim 1, characterized in that, The methods for generating the sound field complexity index include: The covariance matrix of the microphone array is decomposed into eigenvalues, and the ratio of the largest eigenvalue to the trace is extracted as the spatial coherence of the sound field. And calculate the inter-frame difference of the spatial coherence of the sound field as the non-stationary transient rate; The spatial coherence of the sound field and the non-stationary transient rate are fused to generate a quantified sound field complexity index.

4. The method for dynamic optimization of noise suppression and frequency response equalization of a sound system in a complex sound field environment according to claim 1, characterized in that, In step S2, the characteristic-level bidirectional joint solution for noise suppression and dynamic frequency response equalization is performed to generate the initial filter coefficient set, which includes: The acoustic field state vector and the system reference signal are input into a physically constrained adaptive filter for echo cancellation to obtain the residual signal. The residual signal and the sound field state vector are input into a lightweight convolutional recurrent network. The frequency domain physical constraint mechanism and the space-psychoacoustic joint masking surface are combined to perform dual physical embedded constraint extraction processing to obtain the perceptual weighted noise reduction gain flow. Dynamic target frequency response approximation processing is performed based on the broadband room impulse response and background noise floor measured online to obtain the frequency response compensation coefficient flow; The perceptual weighted noise reduction gain stream and the frequency response compensation coefficient stream are subjected to spectral conflict decoupling and alternating iterative projection processing using an alternating projection joint solution module based on spectral masking and compensation constraints to obtain the initial filter coefficient set.

5. The method for dynamic optimization of noise suppression and frequency response equalization of a sound system in a complex sound field environment according to claim 4, characterized in that, The dual physical embedded constraint extraction process includes: The time-varying acoustic boundary condition tensor generated by S1 is used as the frequency domain energy conservation constraint correction term for the room normal mode, and the energy residual of the frequency domain acoustic transfer function is calculated using the frequency domain energy residual surrogate model as the regularization term of the loss function. The masking threshold calculated based on the psychoacoustic model is combined with the three-dimensional coordinates of the audience obtained by millimeter-wave radar to calculate the spatial spectral distortion compensation of the sound field, and a spatial-psychoacoustic joint masking surface is constructed. This surface is then transformed into an adaptive amplitude truncation boundary of the activation function.

6. The method for dynamic optimization of noise suppression and frequency response equalization of a sound system in a complex sound field environment according to claim 4, characterized in that, The process of using an alternating projection joint solution module based on spectral masking and compensation constraints to perform spectral conflict decoupling and alternating iterative projection processing on the perceptual weighted noise reduction gain stream and frequency response compensation coefficient stream includes: A spectrum conflict decoupling gating mechanism is introduced to extract the output gain conflict metric of the weighted noise reduction gain stream and the frequency response compensation coefficient stream in the same frequency band. The cosine similarity of the rate of change of the spectrum envelope of the two streams is calculated. When the similarity is less than 0, it is determined to be a conflict frequency band. Introducing learnable gating scalars The weighted noise reduction gain stream will be output. With frequency response compensation coefficient current output Adaptive interpolation fusion is performed in conflicting frequency bands: ,in This indicates that the shared multilayer perceptron dynamically outputs data based on the current frequency band masking margin and frequency response deficiency. fusion gain Projecting onto the psychoacoustic masking constraint set, i.e., projecting onto the set. Then project it onto the frequency response compensation constraint set. The two-step projection is executed alternately until the preset convergence condition is met; in, This represents the set of filter gains that satisfy psychoacoustic masking constraints. This represents the gain vector of the filter. This represents the gain value in frequency band f. This represents the masking threshold on frequency band f. Let H represent the set of filter gains that satisfy the frequency response compensation constraint, and let H represent the transfer function of the environment. This refers to the frequency response actually heard. This represents the frequency response curve of a dynamic target. Denotes the square of the L2 norm. This indicates the error threshold.

7. The method for dynamic optimization of noise suppression and frequency response equalization of a sound system in a complex sound field environment according to claim 1, characterized in that, The feedforward correction of the initial filter coefficient set based on the sound field state vector and the short-time sound field state prediction network includes: The short-time sound field state prediction network is used as the predictor of the model prediction control. Based on the sound field state vectors of the current frame and historical frames, it predicts the trend of sound field state change in the future preset time delay window. In the prediction time domain, the future control sequence is solved and the first element is taken as the feedforward correction amount. The noise reduction and equalization coefficients are calculated and updated smoothly in advance. The predicted future sound field state vector is fed forward into the joint cost function supernetwork, and the cost function weights are pre-adjusted in advance based on the predicted situation.

8. The method for dynamic optimization of noise suppression and frequency response equalization of a sound system in a complex sound field environment according to claim 7, characterized in that, The construction and updating methods of the joint cost function hypernetwork include: The current frame sound field state vector, the future sound field state vector predicted by the short-time sound field state prediction network, the physical constraint residual of the frequency domain energy residual surrogate model, and the spectral conflict divergence index jointly solved by alternating projection are concatenated and input into a weight generator consisting of three fully connected layers. The output is a normalized […]. , , , Weight vector; Constructed joint cost function: ; in, Indicates the combined value of the system. This represents the weighting coefficient for the mean squared error loss. This represents the mean square error between the system output signal and the target pure signal. This represents the weighting coefficient for the perceived spectral distortion penalty. This represents the perceptual spectral distortion penalty term based on the Buck scale. This represents the weighting coefficient for the spectral flatness penalty. This represents the spectral fluctuation penalty term. This represents the weighting coefficient for hardware dynamic margin constraints. This indicates the power amplifier's thermal protection and clipping distortion limitations; Increase in strong non-stationary disturbance scenarios Increase in deep reverberation standing wave scenarios Increase in advance in dynamic mutation scenarios Dynamic equilibrium occurs when the spectral conflict divergence index exceeds the limit. and The proportions.

9. The method for dynamic optimization of noise suppression and frequency response equalization of a sound system in a complex sound field environment according to claim 8, characterized in that, The dynamic adaptive allocation of network computing power and physics solution accuracy based on the sound field complexity index includes: A first threshold and a second threshold are preset, and the first threshold is less than the second threshold; When the sound field complexity index is below the first threshold, the sound field is determined to be stable. Dynamic channel pruning is performed on the convolutional recurrent network to a preset low power ratio, and a low-order IIR filter or a low-tap FIR filter is used. When the complexity index exceeds the second threshold, a sudden change in the sound field is detected, all network channels are activated, and the system switches to a higher-order FIR filter. The number of spatial grid computation branches activated and the number of alternating projection iterations are dynamically adjusted in step S2 of the frequency domain energy residual proxy model solution based on the computing power budget.

10. The method for dynamic optimization of noise suppression and frequency response equalization of a sound system in a complex sound field environment according to claim 1, characterized in that, The updated filter coefficients are smoothly distributed and iteratively optimized. Simultaneously, the spectral conflict divergence index generated by incremental learning is used as the basis for adjusting the physical prior resolution and fed back to step S1, including: The updated coefficients are smoothly loaded into the digital signal processor using sinusoidal interpolation, and the frequency response deviation and residual noise power error of the output signal are monitored. During the device's idle period, edge incremental learning is performed on a lightweight network, including a front-end feature extraction network and a frequency domain energy residual proxy model, using error data, buffered spectral conflict divergence index, and corresponding acoustic field features. An elastic weight consolidation mechanism is introduced, the Fisher information matrix of network parameters with respect to the historical sound field dataset is calculated, and a parameter importance penalty term is added to the fine-tuning loss function; Based on the accumulated conflict divergence index, the activation strategy of the spatial grid calculation branch of the physical constraint layer of the mid-frequency domain energy residual surrogate model in S2 is adaptively adjusted, and the spectral conflict divergence index is fed back to S1 as the basis for adjusting the spatial resolution weight when the radar point cloud is converted into an impedance scatterer distribution map.

Images