Stop band residual compensation method and device for digital filter, storage medium and computer program product

Abstract

The invention discloses a stop-band residual compensation method and device of a digital filter, a storage medium and a computer program product, and relates to the technical field of audio signal processing, and the method comprises the steps: collecting an original audio signal and a digital filtering signal; the digital filtering signal and the original audio signal are synchronously input into a pre-constructed stop-band residual noise dynamic compensation model to be processed, a smooth transition signal is obtained, and the stop-band residual noise dynamic compensation model takes a hardware filtering signal output by a hardware simulation filter as a target label; and superposing the smooth transition signal and the digital filtering signal according to a preset superposition proportion to obtain a calibration signal. The stop-band residual noise dynamic compensation model learns smooth transition characteristics when the hardware simulation filter reproduces parameter adjustment, and the flexibility of the digital filter is considered, so that the response speed when the digital filter adjusts the cut-off frequency in real time at high frequency is improved, and the stop-band residual energy at the moment of cut-off frequency adjustment is inhibited.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio signal processing, in particular to a stopband residual compensation method and device of a digital filter, a storage medium and a computer program product. BACKGROUND

[0002] Digital filters (such as IIR low-pass or high-pass filters) are widely used in real-time variable cutoff frequency scenarios. When the cutoff frequency needs to be adjusted continuously at a high frequency in a single sample period, the historical state parameters (delay unit register values) inside the filter must be immediately replaced with the values corresponding to the new frequency. Currently, there are two main update strategies: one is to directly clear the old parameters or replace them with new parameters at once; the other is to run two sets of parameters in parallel within a time window of tens of milliseconds and mix the output proportionally before switching. However, the former mathematically destroys the phase continuity between the original state and the new state, causing transient pole drift in the stopband region of the filter transfer function, resulting in rapid rise of stopband energy and formation of high-frequency spikes; the latter suppresses the spikes through delay mixing, but the fixed delay window (≥50 ms) introduced makes the filter unable to respond within the next sample period, thus unable to support high-frequency real-time parameter adjustment.

[0003] Therefore, how to improve the response speed of the digital filter when adjusting the cutoff frequency at a high frequency in real time, and effectively suppress the stopband residual energy at the moment of cutoff frequency adjustment, has become a technical problem that needs to be solved urgently by the present application.

[0004] The above content is only used to assist in understanding the technical solutions of the present application and does not mean that the above content is prior art. SUMMARY

[0005] The main purpose of the present application is to provide a stopband residual compensation method and device of a digital filter, a storage medium and a computer program product, aiming to solve the technical problem of how to improve the response speed of the digital filter when adjusting the cutoff frequency at a high frequency in real time, and effectively suppress the stopband residual energy at the moment of cutoff frequency adjustment.

[0006] To achieve the above purpose, the present application provides a stopband residual compensation method of a digital filter, which comprises: collecting an original audio signal and a digital filter signal; synchronously inputting the digital filter signal and the original audio signal into a pre-constructed stopband residual noise dynamic compensation model for processing to obtain a smooth transition signal, wherein the stopband residual noise dynamic compensation model takes a hardware filter signal output by a hardware analog filter as a target label; stacking the smooth transition signal and the digital filter signal at a preset stacking ratio to obtain a calibrated signal.

[0007] In an embodiment, the step of processing the digital filtering signal and the original audio signal in a pre-constructed stopband residual noise dynamic compensation model to obtain a smooth transition signal further comprises: collecting a training data set based on a pre-deployed hardware digital dual-mode synchronization architecture, a data format of the training data set being pure time domain sampling data, the training data set including a hardware analog filtering signal, an original audio training signal, and a digital filtering training signal; inputting the original audio training signal and the digital filtering training signal into a deep recurrent network model with the hardware analog filtering signal as a target label to generate a preliminary prediction signal; determining a final prediction signal according to the preliminary prediction signal and the digital filtering training signal; calculating a total loss value between the final prediction signal and the target label, updating parameters of the deep recurrent network model based on the total loss value, and iteratively performing the steps of inputting the original audio training signal and the digital filtering training signal into the deep recurrent network model with the hardware analog filtering signal as the target label to generate the preliminary prediction signal until the total loss value reaches loss convergence; under the condition that the total loss value reaches loss convergence, taking the iterated deep recurrent network model as a stopband residual noise dynamic compensation model.

[0008] In an embodiment, the step of collecting a training data set based on a pre-deployed hardware digital dual-mode synchronization architecture further comprises: loading a digital filtering plug-in and an original playback track, and forming a digital plug-in track based on the digital filtering plug-in; connecting a pre-deployed hardware analog filter through an audio interface loop to obtain a hardware processing track; synchronizing cutoff frequency control parameters of the hardware processing track and the digital plug-in track, the cutoff frequency control parameters being obtained after being mapped into control signals based on an automatic curve; building a hardware digital dual-mode synchronization architecture according to the hardware processing track, the digital plug-in track, and the original playback track.

[0009] In an embodiment, the step of calculating a total loss value between the final prediction signal and the target label comprises: adopting mean square error as a time domain loss function and short-time Fourier transform as a frequency domain loss function; calculating a time domain loss value between the final prediction signal and the target label according to the time domain loss function; performing a fast Fourier transform on the final prediction signal and the target label, and calculating a frequency domain loss value of the final prediction signal and the target label after the fast Fourier transform according to the frequency domain loss function; obtaining a total loss value according to the time domain loss value and the frequency domain loss value.

[0010] In an embodiment, the step of calculating the time domain loss value between the final prediction signal and the target label according to the time domain loss function further comprises: obtaining a physical characteristic parameter of a hardware analog filter in the hardware digital dual-mode synchronization architecture; constructing a prior constraint term based on the physical characteristic parameter, and integrating the prior constraint term into the time domain loss function.

[0011] In an embodiment, the step of synchronously inputting the digital filtered signal and the original audio signal into the pre-constructed stopband residual noise dynamic compensation model for processing to obtain a smooth transition signal comprises: synchronously inputting the digital filtered signal and the original audio signal into the pre-constructed stopband residual noise dynamic compensation model at a sample point level for stopband residual noise dynamic compensation, and outputting a smooth transition signal point by point; The stopband residual noise dynamic compensation comprises: performing point-by-point feature capturing on the digital filtered signal and the original audio signal by a multi-level gated recurrent unit of the stopband residual noise dynamic compensation model to obtain transient features; mapping the transient features to compensation coefficients by a fully connected layer of the stopband residual noise dynamic compensation model, and performing dynamic compensation parameter calculation based on the compensation coefficients.

[0012] In an embodiment, the step of synchronously inputting the digital filtered signal and the original audio signal into the pre-constructed stopband residual noise dynamic compensation model for processing to obtain a smooth transition signal further comprises: monitoring a group band frequency band of the smooth transition signal, and calculating an average noise energy value in the group band frequency band based on a short-time Fourier transform; if the average noise energy value is higher than a preset noise energy value in a continuous preset monitoring period, triggering model optimization of the stopband residual noise dynamic compensation model, wherein the step of triggering model optimization of the stopband residual noise dynamic compensation model comprises: extracting a noise sample from the smooth transition signal, and updating a loss function weight of the stopband residual noise dynamic compensation model based on the noise sample; adjusting hyperparameters of the stopband residual noise dynamic compensation model by grid search.

[0013] In addition, to achieve the above object, the present application further provides a device for compensating stopband residue of a digital filter, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the method for compensating stopband residue of a digital filter.

[0014] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the method for compensating stopband residue of a digital filter.

[0015] In addition, to achieve the above object, the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the method for compensating stopband residue of a digital filter.

[0016] The one or more technical solutions provided by the present application have at least the following technical effects: The original audio signal and the digital filter signal are obtained based on a pre-deployed hardware digital dual-mode synchronization architecture; the digital filter signal and the original audio signal are synchronously input into a pre-constructed stopband residue noise dynamic compensation model for processing to obtain a smooth transition signal, and the stopband residue noise dynamic compensation model takes a hardware filter signal output by a hardware analog filter as a target label. The smooth transition signal and the digital filter signal are superimposed at a preset superimposition ratio to obtain a calibration signal. First, the synchronous acquisition of the original audio signal and the digital filter signal is realized by the hardware digital dual-mode synchronization architecture, so as to ensure the non-delayed data acquisition; in addition, the stopband residue noise dynamic compensation model is trained with the smooth transition signal output by the hardware analog filter as the target label, so that the model can learn the natural advantage of the analog filter based on the state continuity of the physical element, thereby accurately reproducing the smooth transition characteristics in the parameter adjustment in the digital domain. Further, the smooth transition signal and the digital filter signal are superimposed at a preset superimposition ratio to retain the high-frequency details after the processing of the digital filter, and at the same time, the calibration signal optimized in consideration of the smooth characteristics is obtained. Therefore, the digital filter signal and the original audio signal are synchronously input into the pre-constructed stopband residue noise dynamic compensation model for processing to finally output the smooth transition signal with non-delayed and effective stopband residue energy suppression, and the flexibility of the digital filter and the low noise performance of the analog system are considered, so as to improve the response speed of the digital filter when adjusting the cutoff frequency in real time, and effectively suppress the stopband residue energy in the instant of adjusting the cutoff frequency. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate preferred embodiments of the present application and, together with the description, serve to explain the principles of the application.

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings required to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, those skilled in the art can obtain other drawings from these drawings without any creative effort.

[0019] Figure 1 The flowchart provided for the first embodiment of the stopband residual compensation method of the digital filter of the present application; Figure 2 The flowchart provided for the second embodiment of the stopband residual compensation method of the digital filter of the present application; Figure 3 The flowchart provided for the third embodiment of the stopband residual compensation method of the digital filter of the present application; Figure 4 The schematic diagram of the hardware digital dual-mode synchronous architecture provided by the present application; Figure 5 The overall flowchart of the stopband residual compensation method provided by the present application; Figure 6 The module structure schematic diagram of the stopband residual compensation device of the digital filter of the embodiment of the present application; Figure 7 The device structure schematic diagram of the hardware running environment involved in the stopband residual compensation method of the digital filter in the embodiment of the present application.

[0020] The purpose implementation, functional features and advantages of the present application will be further explained with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0022] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings in the specification and specific embodiments.

[0023] The main solution of the embodiment of the present application is to build a set of "hardware-digital dual-mode synchronous acquisition architecture" inside the digital audio workstation to acquire the original audio signal X (n) and the preliminary output Y of the digital filter 1(n) , the output Y of the hardware filter (n)The three are taken as a training data set to train a deep recurrent network model to obtain a stopband residual noise dynamic compensation model. Among them, the output of the hardware filter is taken as the target label, the mean square error is taken as the time domain loss function, and the short-time Fourier transform is taken as the frequency domain loss function. After the model is trained, the original audio signal and the digital filter signal are input into the trained model, and the smooth transition signal retaining the smoothing characteristics of the hardware filter can be obtained. At the same time, in order to retain the high-frequency details after the digital filter processing, the calibration signal optimized in consideration of the smoothing characteristics is obtained, the smooth transition signal and the digital filter signal are superimposed with a preset superposition ratio to obtain the calibration signal. Specifically, the model takes a gated recurrent unit GRU as the core, performs forward inference in a single sampling step, and directly outputs the compensated smooth transition signal Y 2(n) , the smooth transition signal and the digital filter signal are superimposed to obtain the final calibration signal Y 3（n） . In the network training phase, the original audio track, the digital plug-in track and the recorded track processed by the hardware analog filter are locked in the same clock domain through the three-track synchronization system of the DAW, and the output Y (n) of the hardware filter is taken as the target label; the network takes the mean square error of Y 3(n) and Y (n) , and the spectral envelope as the target, learns the nonlinear inertia characteristics of the analog filter in the process of capacitor charging and discharging, so as to master how to suppress the stopband transient peak caused by phase discontinuity at the moment of sudden change of cutoff frequency. After training, the neural network model weight is frozen and deployed in the real-time link: when the user changes the cutoff frequency of the digital filter in real time through the MIDI CC (MIDI Continuous Controller), the digital filter immediately outputs Y 1(n) according to the new parameters, and the output and the original signal X (n) are aligned at the sampling stage and enter the network at the same time, and the network is processed in a very low delay, and finally the smooth transition signal Y 2(n) which retains the advantages of flexible editing of digital system and suppresses the stopband residual energy to below-70 dB is obtained, realizing the non-delay smooth transition and stopband residual suppression of the digital filter when the cutoff frequency is adjusted in real time at high frequency.

[0024] The embodiments of the present application consider that: first, the digital filter has the following problems when adjusting the cutoff frequency quickly: when adjusting the cutoff frequency of the low-pass / high-pass filter in real time (for example, voice enhancement in audio production or special sound effect scene), the historical state (delay unit value) parameters stored in the mathematical calculation formula inside the filter need to be updated synchronously. However, if the old parameters are directly cleared or forcibly overwritten, it will cause transient oscillation and step mutation of the output signal-for example, the music being played is suddenly "interrupted" and then connected, producing a short "burst" or noise. The root cause of this problem is that the digital filter relies on accurate mathematical calculation and parameter preset, and rough parameter switching may cause short but irritating high-frequency noise. The essence is that the phase continuity of the mathematical model is destroyed: there is a lack of smooth transition between the waveform memory (such as resonance energy) accumulated by the original filter structure and the calculation state of the new parameter system. Through spectral analysis, it can be seen that there are obvious noise signal residues in the stopband region.

[0025] To solve the problem of burst noise residue when adjusting the cutoff frequency of the digital filter (such as IIR low-pass or high-pass) in real time, engineers often use gradual parameter transition: first, retain part of the old parameter calculation weight, and then gradually transition to the new parameter. The system runs two sets of filter parameters at the same time, and mixes the output signals in proportion within tens of milliseconds. For example, when switching the voice enhancement mode in real time in a karaoke APP, the algorithm will temporarily retain the old parameter's suppression effect on high frequencies, while gradually increasing the new parameter's attenuation on low frequencies to avoid "puffing" switching noise.

[0026] Another method is to split the frequency adjustment into multiple small changes (such as adjusting from 100Hz to 200Hz ten times), which can reduce transient noise, similar to climbing stairs rather than jumping high, but at the cost of increased adjustment delay (for example, the filter response will be slow by 0.1 seconds) and reduced response speed, which cannot meet the needs of some real-time scenarios. This method is commonly used in embedded microphone noise reduction chips, which sacrifices about 50ms of delay for a frequency adjustment process without burst noise.

[0027] The above optimization strategies can alleviate or mask the signal mutation problem caused by parameter update of the digital filter to some extent, but they all introduce a certain time delay, which sacrifices the adjustment speed (about 100ms) for non-mutation switching. This limits the application in real-time high-frequency parameter adjustment scenarios.

[0028] Further, analog filters are based on the voltage and current changes of physical components such as resistors and capacitors, which naturally avoids the problem of parameter mutation. The core advantage of analog filters lies in the continuity of the state of physical components. For example, when adjusting the cutoff frequency of the Moog ladder filter in a classic synthesizer, the charge inside the capacitor will not reset instantly, but will gradually transition to the new state according to the physical state of the component, without the "parameter conflict" caused by mathematical truncation errors in digital systems. Just like a spring gradually returns to its original state after being compressed, it slowly migrates to the new state. This "inertia buffer" given by the physical properties of electronic components (such as capacitor charging time and inductor hysteresis effect) makes the sound change as natural and smooth as water flowing from a narrow pipe into a wide channel. Even if the knob is quickly twisted, the sound change can still remain soft and natural.

[0029] However, it is difficult to integrate analog systems into modern digital audio workflows. For example, adding an analog filter effect to a pre-recorded guitar audio in a DAW (Ableton Live or ProTools) requires reconnection of the hardware, playback, and recording of the entire audio track every time the parameters of the analog filter are modified. The original signal cannot be restored by undoing, and the efficiency is extremely low. In addition, the characteristic deviation of analog circuits (such as frequency response shift caused by capacitor aging) will make it difficult to reproduce the effects of the same set of knob parameters at different time periods, which is not conducive to music production scenarios that require precise replication of effects. Digital filters can not only repeatedly listen to the effects of different cutoff frequencies on the same audio segment, but also precisely control the parameter change trajectory by drawing automation curves (for example, allowing the low-pass frequency to fluctuate with the beat rhythm), which is an efficiency advantage that analog devices cannot achieve.

[0030] Therefore, the present application provides a solution that preserves the flexibility of digital filter processing while replicating the organic response characteristics of analog filters, thereby improving the response speed of digital filters when adjusting the cutoff frequency at high frequencies in real time, and suppressing the residual energy in the stopband during the adjustment of the cutoff frequency, effectively suppressing step noise.

[0031] Specifically, the original audio signal and the digital filter signal are acquired based on a pre-deployed hardware digital dual-mode synchronization architecture; the digital filter signal and the original audio signal are synchronously input into a pre-constructed stopband residual noise dynamic compensation model for processing to obtain a smooth transition signal, the stopband residual noise dynamic compensation model taking a hardware filter signal output by a hardware analog filter as a target label. The smooth transition signal and the digital filter signal are superimposed at a preset superimposition ratio to obtain a calibration signal. First, the original audio signal and the digital filter signal are synchronously acquired through the hardware digital dual-mode synchronization architecture to ensure delay-free data acquisition; in addition, the stopband residual noise dynamic compensation model is trained with the smooth transition signal output by the hardware analog filter as the target label, so that the model can learn the natural advantage of the analog filter based on the continuity of the physical element state, thereby accurately reproducing the smooth transition characteristics during parameter adjustment in the digital domain. Further, the smooth transition signal and the digital filter signal are superimposed at a preset superimposition ratio to retain the high-frequency details after digital filter processing while obtaining the calibration signal that takes into account the optimized smoothness characteristics. Therefore, the digital filter signal and the original audio signal are synchronously input into the pre-constructed stopband residual noise dynamic compensation model for processing to finally output the smooth transition signal that is delay-free and effectively suppresses the stopband residual energy, taking into account the flexibility of the digital filter and the low-noise performance of the analog system, thereby improving the response speed of the digital filter when adjusting the cutoff frequency in real time and effectively suppressing the stopband residual energy at the moment of adjusting the cutoff frequency.

[0032] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a stopband residual compensation system, etc. that can realize the above functions. The following will take the stopband residual compensation system as an example to describe the embodiment and the following embodiments.

[0033] Based on this, the embodiment of the present application provides a stopband residual compensation method of a digital filter, which refers to Figure 1 , Figure 1 The flowchart of the first embodiment of the stopband residual compensation method of the digital filter of the present application is shown in the figure.

[0034] In the embodiment, the stopband residual compensation method of the digital filter includes steps S10-S30: Step S10, acquiring an original audio signal and a digital filter signal; The original audio signal refers to a pulse code modulation (PCM) data stream stored in a digital audio workstation (DAW) in WAV or AIFF format, which is not processed by any filter and retains full-band information; the digital filtered signal refers to the output data stream of the original audio signal after preliminary processing by an IIR biquad digital filter, which contains residual noise in the stop band due to real-time parameter adjustment.

[0035] The acquisition process is realized through the audio driver layer of the DAW. The system calls the ASIO or Core Audio interface to read the original audio signal from the track buffer frame by frame, and simultaneously reads the digital filtered signal from the output buffer of the digital filter plug-in, thereby ensuring that the two signals are aligned at the sampling clock level.

[0036] It should be noted that the digital filter plug-in in the training phase and the digital filter in the inference phase of the present application both use an IIR biquad filter. Its transfer function is:

[0037] wherein, , , , , is the filter coefficient.

[0038] Step S20, synchronously inputting the digital filtered signal and the original audio signal into a pre-constructed stop-band residual noise dynamic compensation model for processing to obtain a smooth transition signal, the stop-band residual noise dynamic compensation model taking the hardware filtered signal output by the hardware analog filter as a target label.

[0039] The stop-band residual noise dynamic compensation model is a supervised learning model taking a deep recurrent network as the main body and taking the hardware filtered signal output by the hardware analog filter as the target label. The input end simultaneously receives the digital filtered signal and the original audio signal, and the output end generates a smooth transition signal.

[0040] The smooth transition signal is the final audio sample sequence that eliminates the popping sound caused by parameter mutation in the digital domain and approximates the characteristics of the hardware filter in phase continuity.

[0041] Synchronous input: after strictly aligning the digital filtered signal and the original audio signal on the time axis at the sampling point level, feed them into the model point by point to meet the streaming real-time processing requirements.

[0042] In a possible implementation, the deep recurrent network can be selected as a 4-layer bidirectional GRU (Gated Recurrent Unit) with a hidden dimension of 256 in each layer, and a 1x1 convolution is performed after each layer for channel mixing. Specifically, the deep recurrent network with 4 layers of bidirectional GRU is used, and the hidden dimension of each layer is set to 256, that is, each layer contains 256 hidden units. The low-level to high-level bidirectional time sequence features are extracted through the multi-layer bidirectional GRU, and the complex dynamics of the transient signal are captured. Then, the 1x1 convolution is performed for channel mixing, the feature channels output by different hidden layers are fused, and the modeling capability of the transient signal such as the stopband residual noise is further enhanced.

[0043] In another possible implementation, a causal Transformer architecture can also be used in cooperation with the rotary position encoding to further capture the long-range transient dependence. Specifically, the causal Transformer architecture is used in cooperation with the rotary position encoding. The causal architecture ensures that the model only depends on the past time step information to meet the real-time processing requirement. The rotary position encoding integrates the position information into the feature vector through a rotation matrix, solves the limitation of the Transformer in modeling the long sequence position, and thus more accurately captures the long-range transient dependence of the parameter switching in the audio signal, and improves the stability and continuity of the stopband residual compensation.

[0044] In step S30, the smooth transition signal and the digital filter signal are superimposed at a preset superimposition ratio to obtain a calibration signal.

[0045] The preset superimposition ratio refers to a weight a(n) continuously changing between 0 and 1, which is predicted by the system in real time according to the neural network and acts between the smooth transition signal and the digital filter signal. The calibration signal refers to the audio data stream finally output to the DAW main output or the loudspeaker driver after the a(n) weighted superimposition.

[0046] First, the smooth transition signal and the digital filter signal are aligned sample by sample, and then vector operation is performed: The calibration signal Y 3(n) = a(n) Y 2(n) + (1-a(n)) Y 1(n) wherein Y 2(n) represents the smooth transition signal, Y 1(n) represents the digital filter signal, and a(n) represents the preset superimposition ratio. a(n) is much greater than 1-a(n), and the purpose is to retain the smooth transition signal in a larger proportion.

[0047] The embodiment provides a stopband residual compensation method of a digital filter, synchronization acquisition of an original audio signal and a digital filter signal is realized through a hardware digital dual-mode synchronization architecture, and delay-free data acquisition is ensured; in addition, a stopband residual noise dynamic compensation model is trained with a smooth transition signal output by a hardware analog filter as a target label, so that the model can learn the natural advantage of the analog filter based on the continuity of the physical element state, and thus the smooth transition characteristics during parameter adjustment are accurately reproduced in the digital domain. Further, the smooth transition signal and the digital filter signal are superimposed at a preset superposition ratio, the high-frequency details after digital filter processing are reserved, and at the same time, the calibration signal considering the optimized smoothness characteristics is obtained. Therefore, the digital filter signal and the original audio signal are synchronously input into the pre-constructed stopband residual noise dynamic compensation model for processing, and finally the smooth transition signal with no delay and effectively suppressed stopband residual energy is output, the flexibility of the digital filter and the low-noise performance of the analog system are considered, so that the response speed of the digital filter when adjusting the cutoff frequency in real time is improved, and the stopband residual energy at the moment of adjusting the cutoff frequency is effectively suppressed.

[0048] Based on the first embodiment of the application, the second embodiment of the application is proposed. In the second embodiment of the application, the same or similar contents as the above first embodiment can be referred to the above introduction, and the subsequent will not be described in detail.

[0049] On this basis, please refer to Figure 2 , Figure 2 The flowchart of the second embodiment of the application is shown as Figure 2 Before step S20, steps S011-S015 are further included. In step S011, a pre-deployed hardware digital dual-mode synchronization architecture is used to collect a training data set, the data format of the training data set is pure time domain sampling data, and the training data set includes a hardware analog filter signal, an original audio training signal and a digital filter training signal. The pre-deployed hardware digital dual-mode synchronization architecture: a closed-loop signal link composed of a digital audio workstation (DAW), an audio interface, a hardware analog filter and a synchronous clock source, which ensures that the sampling clock of the digital domain and the analog domain is strictly aligned through MIDI clock or word clock (WordClock).

[0050] The training data set refers to the entire data set used for subsequent supervised learning, and the data format is pure time domain sampling data, that is, each sample point only contains time sequence amplitude values arranged according to the sampling rate, and does not contain any frequency domain or transform domain information.

[0051] The hardware analog filter signal refers to the waveform data obtained by processing the original audio training signal through a real hardware analog filter and sampling it back through an audio interface. The waveform data has capacitance charging and discharging inertia and can be used as an ideal reference without burst noise. The original audio training signal refers to an input audio segment used to drive a digital filter plug-in and a hardware analog filter during the training phase. The original audio training signal usually includes white noise sweep, music segments, impulse signals, etc. The digital filter training signal refers to the output waveform obtained by processing the original audio training signal in real time through a digital filter plug-in. The waveform retains the residual noise in the stopband caused by parameter jumps.

[0052] The system plays the original audio training signal through track 0 of the DAW, loads the digital filter plug-in to obtain the digital filter training signal through track 1, and connects the hardware analog filter through the audio interface loop to obtain the hardware analog filter signal through track 2. The three-track data is written into the training data set frame by frame.

[0053] In step S012, the original audio training signal and the digital filter training signal are input into a deep recurrent network model with the hardware analog filter signal as the target label to generate a preliminary prediction signal. The deep recurrent network model refers to a sequence-to-sequence network composed of multiple levels of gated recurrent units (GRU). The hidden state dimension, number of layers, and dropout ratio have been determined in the hyperparameter search phase. The preliminary prediction signal refers to the first estimate of the ideal noise-free output under the current parameter condition. The estimate has not been fused with the digital filter training signal. The target label is the aforementioned hardware analog filter signal.

[0054] The original audio training signal and the digital filter training signal are spliced by sample points to form a two-dimensional input feature vector (channel 0 is the original audio training signal and channel 1 is the digital filter training signal), which is input into the deep recurrent network model. The model captures transient characteristics and capacitance inertia rules through GRU units and finally generates a preliminary prediction signal with the same length as the target label in the output layer. In addition, it should be noted that in one possible implementation, the deep recurrent network model can be stacked with an attention mechanism based on bidirectional GRU to enhance the positioning ability of the mutation interval.

[0055] In step S013, the final prediction signal is determined based on the preliminary prediction signal and the digital filter training signal. The final prediction signal refers to the compensation result obtained by further fusing the digital filter training signal based on the preliminary prediction signal. The result is directly used as the loss calculation object. The fusion strategy uses a residual connection method, i.e., adding or weighted adding the digital filter training signal and the preliminary prediction signal sample by sample to retain the high-frequency details of the digital system and suppress the residual noise in the stopband.

[0056] First, the residual energy between the preliminary prediction signal and the digital filter training signal is calculated, and then the fusion of the two is performed according to the preset coefficient λ∈[0, 1]. It can be understood that the preset superposition ratio α(n) is the optimal solution of the preset coefficient λ∈[0, 1], and the preset coefficient λ can be continuously adjusted during the model training process, and finally the preset superposition ratio α(n) used in the prediction process using the model is obtained.

[0057] Step S014, calculate the total loss value between the final prediction signal and the target label, update the parameters of the deep recurrent network model based on the total loss value, and iteratively execute the steps of inputting the original audio training signal and the digital filter training signal into the deep recurrent network model with the hardware simulation filter signal as the target label, generating a preliminary prediction signal, until the total loss value reaches loss convergence. The total loss value is obtained by weighted sum of the time domain loss value and the frequency domain loss value, wherein the time domain loss value adopts mean square error (MSE) to measure the sample point difference between the final prediction signal and the target label, and the frequency domain loss value adopts the amplitude spectrum MSE after short-time Fourier transform (STFT). Loss convergence refers to that the relative decrease of the total loss value in continuous N iterations is less than a preset threshold ε.

[0058] Specifically, the system first calculates the time domain loss, then calculates the amplitude spectrum MSE by respectively performing STFT on the final prediction signal and the target label, and finally sums the total loss value according to the preset weight, updates all trainable parameters of the deep recurrent network model using the back propagation algorithm, and repeats the above process until convergence.

[0059] Step S015, under the condition that the total loss value reaches loss convergence, the iterated deep recurrent network model is taken as the stopband residual noise dynamic compensation model.

[0060] The stopband residual noise dynamic compensation model refers to the deep recurrent network model with the smallest stopband residual noise obtained after iterative training. The model solidifies the weight in ONNX format and is deployed to the inference engine for real-time inference stage.

[0061] After completing the last weight update, the system serializes the network structure, weight matrix, hyperparameter and input / output specification into the stopband residual noise dynamic compensation model, and registers the model in the hardware abstraction layer for subsequent real-time calling.

[0062] In this embodiment, under the condition of alignment at the sampling clock level by the hardware digital dual-mode synchronous architecture, the triple synchronous data of "original audio training signal-digital filter training signal-hardware analog filter signal" is collected, and the hardware analog filter signal is taken as the ideal target label without burst noise; then, taking the deep recurrent network model as the core, the joint loss of time domain and frequency domain is used in the end-to-end supervised learning framework to repeatedly iterate until convergence, and the network obtained by training is solidified as a stopband residual noise dynamic compensation model. Therefore, the system not only inherits the advantages of non-destructive editing, automatic curve control and repeatability of digital filters in the DAW environment, but also reproduces the burst-free and soft listening of hardware analog filters due to the inertia of capacitor charging and discharging. In the real-time cutoff frequency jump scene, the stopband residual noise energy is below -70 dB, and there is no additional audible delay, so as to ensure low delay and low power consumption while significantly improving audio quality and user experience.

[0063] Based on the first embodiment and / or the second embodiment of the present application, the third embodiment of the present application is proposed. In the third embodiment of the present application, the same or similar contents as the above embodiments can be referred to the above introduction, and the subsequent will not be described in detail.

[0064] On this basis, please refer to Figure 3 , Figure 3 The flowchart of the third embodiment of the present application is shown as Figure 3 Based on the pre-deployed hardware digital dual-mode synchronous architecture, the steps S011 of obtaining the training data set before the steps S1-S4 are included: Step S1, load the digital filter plug-in and the original playback track, and form a digital plug-in track based on the digital filter plug-in; Digital filter plug-in: an audio signal processing module realized in software form inside the digital audio workstation, whose functions include but are not limited to low-pass, high-pass, band-pass or band-stop filtering, and usually uses IIR or FIR difference equation structure inside.

[0065] Original playback track: an audio track in a digital audio workstation used to carry original audio signals without any filtering processing. The audio segments in this track can be white noise, music, pulses or any time-domain signals for training and testing.

[0066] Digital plug-in track: an audio track independently created in a digital audio workstation, which only mounts a digital filter plug-in and takes the output of the original playback track as its input, thereby forming a signal processing link in the digital domain.

[0067] Load digital filter plug-in and original playback track, establish a repeatable, editable, and automated digital filter path for subsequent synchronous acquisition, so that the digital filter signal and the subsequent hardware filter signal can be recorded under the same control parameter track, thereby ensuring the timing consistency of the training data.

[0068] In one possible implementation, the parametric equalizer built-in in the audio workstation can be used in combination with the visual programming extension device to map the hidden parameters to MIDI CC signals, realize an additional parameter exposure interface, and then perform MIDI CC mapping.

[0069] Step S2, connect the pre-deployed hardware analog filter through the audio interface loop to obtain a hardware processing track; Audio interface loop: a closed physical link formed by connecting the analog output port and the analog input port of the audio interface through an external hardware analog filter.

[0070] Hardware analog filter: an analog filter circuit composed of active or passive elements such as resistors, capacitors, and inductors, and working in continuous time mode.

[0071] Hardware processing track: an audio track in the digital audio workstation used to record the output signal of the hardware analog filter, with the input end connected to the analog input port of the audio interface.

[0072] The hardware processing track obtained by connecting the pre-deployed hardware analog filter through the audio interface loop introduces the physical characteristics of the real analog system such as nonlinearity, phase drift, and transient inertia into the data acquisition link, so that the subsequent model can learn the "organic" response characteristics that the digital filter cannot directly produce.

[0073] Step S3, synchronize the cutoff frequency control parameters of the hardware processing track and the digital plug-in track, which are obtained after the automatic curve is mapped to a control signal; Cutoff frequency control parameter: a single scalar value used to continuously control the turning frequency of the filter, usually in hertz (Hz).

[0074] Automatic curve: an editable curve in the digital audio workstation with time axis as the horizontal coordinate and parameter value as the vertical coordinate, which can be manually drawn or written in real time based on an external MIDI controller.

[0075] Control signal: a MIDI continuous controller generated by sampling the automatic curve, used to drive the synchronous change of hardware and software filters in real time.

[0076] The purpose of synchronizing the cutoff frequency control parameters of the hardware processing track and the digital plug-in track is to drive the digital plug-in and the hardware analog filter simultaneously with the same control signal, ensuring that their cutoff frequencies remain consistent at any time, thereby making the subsequently acquired digital filtered signal and hardware filtered signal strictly comparable.

[0077] Step S4: Build a hardware digital dual-mode synchronous architecture based on the hardware processing track, the digital plug-in track, and the original playback track.

[0078] A complete acquisition framework is formed by the original playback track, digital plugin track, hardware processing track, audio interface, hardware analog filter, and unified clock source, resulting in a hardware-digital dual-mode synchronous architecture. In this architecture, the digital signal chain and analog signal chain operate in parallel, sharing the sampling clock, MIDI clock, and automation control clock. This integrates the aforementioned three tracks and their external devices into a closed-loop system capable of simultaneously outputting the original audio signal, digitally filtered signal, and hardware-filtered signal with millisecond-level time precision, providing rigorously aligned data for subsequent deep learning training.

[0079] Additionally, it should be noted that in one possible implementation, the unified clock source can be provided by the WordClock output of the audio interface, and then distributed to the clock input of the digital workstation and external hardware via a BNC cable (Bayonet Neill-Concelman Cable). In another possible implementation, Ethernet clock synchronization based on the Ethernet high-precision time synchronization protocol can also be used to keep the entire system consistent in a distributed recording studio environment.

[0080] refer to Figure 4 , Figure 4 This is a schematic diagram of the hardware digital dual-mode synchronous architecture provided in this application. Figure 4 As shown, a hardware-digital dual-mode synchronous data acquisition architecture is built, a three-track synchronous system is constructed in a digital audio workstation, the automated curve is mapped to a MIDI CC control signal, and millisecond-level dynamic parameter tuning is achieved by binding filter cutoff frequency control parameters.

[0081] In the embodiment, a clock-unified, parameter-consistent, and signal-parallel "hardware-digital dual-mode synchronous architecture" is constructed between the internal and external analog hardware of the digital audio workstation. The architecture can synchronously output three sample-level aligned audio streams, i.e., an unprocessed original audio signal, a digital filter signal processed by a digital filter plug-in, and a hardware filter signal processed by a hardware analog filter. Thus, all advantages of the digital system, such as editability, reversibility, and automation, are retained, and the "organic" transient characteristics caused by the inertia, nonlinearity, and phase drift of the analog system are completely captured. High-precision, strictly aligned, and rich-physical-behavior training and verification data are provided for a subsequent dynamic compensation model of residual noise in the stopband, thereby fundamentally solving the problem of coexistence of residual burst and response delay during real-time parameter adjustment of the digital filter.

[0082] Based on the above-mentioned embodiments of the present application, a fourth embodiment of the present application is proposed. In the fourth embodiment of the present application, the same or similar contents as the above-mentioned embodiments can be referred to the above introduction, and the subsequent will not be described in detail.

[0083] In the embodiment, the step S014 of calculating the total loss value between the final prediction signal and the target label includes steps S41-S44: In step S41, the mean square error is used as the time-domain loss function, and the short-time Fourier transform is used as the frequency-domain loss function. The mean square error refers to the arithmetic mean of the square of the difference between the prediction sequence and the true value sequence at each sampling point. This index quantifies the overall deviation of the time-domain waveform, and the smaller the value, the higher the reconstruction accuracy. The short-time Fourier transform refers to the discrete Fourier transform performed on each frame after windowing and framing the signal to obtain the amplitude spectrum and phase spectrum that vary with time. This transform can explicitly present the transient error in the frequency domain. The frequency-domain loss function refers to a function that takes the amplitude spectrum after the short-time Fourier transform as input and calculates the difference between the predicted spectrum and the target spectrum in the form of mean square error, which is used to capture the residual energy in the stopband and harmonic distortion.

[0084] By simultaneously configuring the time-domain loss function and the frequency-domain loss function, the joint optimization of "waveform fidelity" and "listening experience fidelity" is realized, so that the network can maintain smooth time waveform and suppress abnormal frequency band protrusions during the training phase.

[0085] Specifically, the time-domain loss function formula is as follows:

[0086] wherein, represents the target label, i.e., the hardware filter signal; is a smooth transition signal output by the model; The frequency-domain loss function formula is as follows:

[0087] wherein, represents the mean value of all elements of the tensor; represents the predicted spectral envelope of the smoothed transition signal after fast Fourier transform; represents the expected spectral envelope of the target label after fast Fourier transform.

[0088] Therefore, the total loss function of the present application is as follows:

[0089] wherein, the time domain weight coefficient , the frequency domain weight coefficient In a specific embodiment, the time domain weight coefficient takes a value of 0.35, and the frequency domain weight coefficient takes a value of 0.65.

[0090] Step S42, calculating the time domain loss value between the final predicted signal and the target label according to the time domain loss function; The time domain loss value refers to the mean square error value between the final predicted signal and the target label, which is obtained by calculating the square of the difference value point by point and averaging, and can directly reflect the severity of the time domain defects such as transient burst and phase misplacement.

[0091] The final predicted signal refers to the network output waveform after residual fusion; and the target label refers to the hardware filter signal output by the hardware analog filter.

[0092] The system obtains the scalar loss value in real time by calculating the mean square error value between the final predicted signal and the target label, and uses the loss value to update the gradient of the network weight in the back propagation stage, so as to gradually reduce the waveform error.

[0093] Step S43, performing fast Fourier transform on the final predicted signal and the target label, and calculating the frequency domain loss value of the final predicted signal and the target label after the fast Fourier transform according to the frequency domain loss function; First, the final predicted signal and the target label are sent into the same fast Fourier transform engine to obtain complex spectrum, and then the amplitude spectrum is extracted to calculate the specific frequency domain loss value according to the frequency domain loss function formula.

[0094] Step S44, obtaining the total loss value according to the time domain loss value and the frequency domain loss value.

[0095] The total loss value refers to the time domain loss value and the frequency domain loss value according to the time domain weight coefficient , the frequency domain weight coefficient a scalar after weighted summation, which is the only basis for network backpropagation, is used to drive all trainable parameters to update in the direction of reducing the overall error; time domain weight coefficient frequency domain weight coefficient In this embodiment, the time domain is 0.35 and the frequency domain is 0.65, which highlights the frequency domain listening experience characteristics, but can be dynamically adjusted according to the subjective score of the verification set. After the system obtains the total loss value by linear combination, it inputs it into the optimizer for gradient descent until the relative decline of the total loss value in continuous multiple iterations is less than the convergence threshold, thereby completing the training of the stopband residual noise dynamic compensation model.

[0096] In this embodiment, the mean square error is used to accurately measure the time domain waveform difference, the short-time Fourier transform amplitude spectrum mean square error is used to capture the frequency domain residual energy, and the two are fused into a single total loss value according to the preset weight; in backpropagation, the total loss value drives all parameters of the deep recurrent network model to continuously update in the gradient direction until the relative decline after continuous multiple iterations is lower than the convergence threshold. Thus, the network can ensure both smoothness of the time domain waveform and deep suppression of the frequency domain stopband energy during the training phase, and the stopband residual noise dynamic compensation model finally converged has the ability to minimize the stopband residual noise, providing a high-precision compensation basis for the subsequent real-time inference phase that takes into account listening experience fidelity and low latency.

[0097] In a possible implementation, the step S42 further includes steps S0231-S0232 before it: In step S0231, the physical characteristic parameters of the hardware analog filter in the hardware digital dual-mode synchronous architecture are obtained. The physical characteristic parameters of the hardware analog filter in the hardware digital dual-mode synchronous architecture: inside the hardware analog filter, a set of measurable physical quantities that can quantitatively describe the transient and steady-state behavior are determined by physical elements such as resistance, capacitance, inductance, operational amplifier, transistor, etc.; specifically including but not limited to gain coefficient of cutoff frequency varying with control voltage, capacitance charging and discharging time constant, inductance hysteresis loop slope, operational amplifier open-loop gain-bandwidth product, transistor transconductance nonlinearity coefficient, and temperature drift compensation coefficient.

[0098] The purpose of obtaining the physical characteristic parameters is to introduce the inertia, nonlinearity, temperature drift, etc. of the real analog circuit into the subsequent loss function in numerical form, thereby constraining the weight update direction of the deep recurrent network, so that the network output not only statistically approximates the hardware filtering signal, but also physically reflects the "continuity" nature that the capacitance charge cannot be reset instantaneously and the inductance current cannot jump.

[0099] The system injects step, sinusoidal sweep and pulse sequences into the hardware analog filter through an automated test script, records the response using a measurement microphone or loopback, and then extracts the above-mentioned physical characteristic parameters through least squares fitting or system identification algorithm to form a structured numerical table or continuous function expression, providing a data basis for the construction of subsequent prior constraint terms.

[0100] Step S0232, constructing a prior constraint term based on the physical characteristic parameters, and integrating the prior constraint term into the time-domain loss function.

[0101] The prior constraint term refers to a differentiable regularization term constructed based on the physical characteristic parameters, which can be in the form of a capacitance discharge exponential decay function, a cutoff frequency smoothness penalty term or a resonance peak damping constraint, used to inject the physical inertia prior of the analog system into the deep recurrent network model during the training phase; the time-domain loss function refers to the mean square error or its weighted variant, used to measure the point-by-point difference between the final predicted signal and the target label. The system embeds the prior constraint term in the time-domain loss function in an additive manner.

[0102] Based on the above embodiments of the present application, the fifth embodiment of the present application is proposed, wherein the same or similar contents as the above embodiments can be referred to the above introduction, and the following will not be described in detail.

[0103] In this embodiment, the step S20 of synchronously inputting the digital filter signal and the original audio signal into the pre-constructed stopband residual noise dynamic compensation model to obtain a smooth transition signal can include the step S21: Step S21, synchronously inputting the digital filter signal and the original audio signal into the pre-constructed stopband residual noise dynamic compensation model at the sampling point level to perform stopband residual noise dynamic compensation, and outputting a smooth transition signal point by point; Sampling point level synchronization: The digital filter signal and the original audio signal are input in parallel point by point in one-to-one correspondence under the same sampling clock driving.

[0104] Stopband residual noise dynamic compensation: The model dynamically calculates the compensation amount for each sampling point and superimposes it in real time, thereby processing the process of suppressing the stopband energy below the target level.

[0105] Smooth transition signal: The output sampling sequence after compensation eliminates the burst noise and maintains the natural transient characteristics of the analog filter.

[0106] Using the trained network to correct the real-time digital filtering result point by point, so that the final output audio approximates the hardware analog filter in phase continuity, energy release and listening softness, while completely retaining the editability and low delay advantage of the digital system.

[0107] In an implementable embodiment, the stopband residual noise dynamic compensation comprises steps S211-S212: Step S211, point-by-point feature capturing by a multi-level gated recurrent unit of the stopband residual noise dynamic compensation model on the digital filtered signal and the original audio signal, to obtain transient features; Multi-level gated recurrent unit: a recurrent neural network module stacked by two or more layers of gated recurrent units, each layer containing an update gate, a reset gate and a candidate hidden state, for capturing short-term transients and long-term dependencies in the time dimension.

[0108] Point-by-point feature capturing: calculating the corresponding hidden state vector for each input sample point, thereby extracting features reflecting transient amplitude, slope and energy change law.

[0109] Transient features: high-dimensional vectors output by the multi-level gated recurrent unit, which can characterize the current sampling point and its local dynamic characteristics.

[0110] The purpose of this step is to fuse the full-band information provided by the original audio signal and the defect information exposed by the digital filtered signal into a unified transient representation, providing a local and global basis for subsequent compensation coefficient calculation.

[0111] In addition, it should be noted that in one possible embodiment, the multi-level gated recurrent unit can use a bidirectional structure to utilize past and future contexts simultaneously; in another possible embodiment, a 1x1 convolution can be added at the front end to mix channels before feeding into a unidirectional gated recurrent unit, to reduce computational complexity.

[0112] Step S212, mapping the transient features to compensation coefficients by a fully connected layer of the stopband residual noise dynamic compensation model, and performing dynamic compensation parameter calculation based on the compensation coefficients.

[0113] Fully connected layer: a linear transformation layer with all input and output neurons connected to each other without weight sharing, which compresses or maps high-dimensional transient features to the target dimension.

[0114] Compensation coefficient: a scalar or vector output by the fully connected layer, used for amplitude or phase correction of the current digital filtered signal sample point.

[0115] Dynamic compensation parameter calculation: real-time calculation of the correction amount that can be superimposed or convolved according to the compensation coefficient, and immediate application to the current sampling point to offset the stopband residual noise.

[0116] The purpose of this step is to convert the transient features into compensating actions that can be directly executed, so that the final output of the smooth transition signal is consistent with the hardware analog filter in both energy envelope and phase trajectory.

[0117] In addition, it should be noted that, in a possible implementation, the full connection layer outputs a single-channel compensation coefficient, which is corrected by simple multiplication; in another possible implementation, a double-channel coefficient can also be output to control the amplitude and phase respectively, and more precise frequency domain correction can be realized through complex multiplication.

[0118] In the embodiment, by synchronously inputting at the sampling point level, the multi-stage gating recurrent unit is used to capture the transient characteristics in real time, and the full connection layer is used to generate compensation coefficients, and the smooth transition signal is output point by point, so that the digital filter burst noise is eliminated while the soft transient of the analog filter is retained, and real-time compensation with low delay and high fidelity is realized.

[0119] Based on the above-mentioned embodiments of the present application, a sixth embodiment of the present application is proposed, and the same or similar contents as the above-mentioned embodiments can be referred to the above description, and will not be described in detail hereinafter.

[0120] In the embodiment, after the step S20 of processing the digital filter signal and the original audio signal synchronously in the pre-constructed stopband residual noise dynamic compensation model to obtain a smooth transition signal, the steps A30-A40 are further included: Step A30, monitoring a group band frequency range of the smooth transition signal, and calculating an average noise energy value in the group band frequency range based on a short-time Fourier transform; Smooth transition signal: an audio output stream after double processing of a digital filter and a neural network compensation filter, which retains the flexibility of real-time parameter adjustment of a digital system, and suppresses the stopband residual noise through a neural network, so as to approach the natural transition of an analog filter in hearing.

[0121] Group band frequency range: a frequency band range extending 1 / 3 octave bandwidth on both sides of the target cutoff frequency as the center, which is used to capture residual noise that may not be completely offset by the neural network at the moment of digital parameter switching.

[0122] Average noise energy value: an energy density average value obtained by integrating the amplitude spectrum squared obtained by the short-time Fourier transform in the group band frequency range and dividing by the frequency band width, and the numerical value directly reflects the strength of the stopband residual noise.

[0123] The smooth transition signal is segmented according to a preset frame length (such as 1024 sampling points) and a preset window function (such as a Hanning window), then a discrete Fourier transform is performed on each frame of signal to obtain a frequency domain representation, and the spectrum is continuously calculated through a sliding window to obtain a short-time Fourier transform value. After the amplitude spectrum squared obtained by the short-time Fourier transform is integrated and divided by the frequency band width, an average noise energy value is obtained.

[0124] The core purpose of real-time monitoring of the group band frequency range is to provide quantitative basis for subsequent model optimization: when the average noise energy value is continuously high, it indicates that the current neural network has insufficient compensation ability for mutation interval, and further fine-tuning is needed; on the contrary, if the energy value is lower than the preset threshold for a long time, it indicates that the model has fully learned the hardware inertia characteristics, and the current weight can be maintained.

[0125] Step A40, if the average noise energy value is higher than the preset noise energy value in the continuous preset monitoring period, triggering the model optimization of the stopband residual noise dynamic compensation model; The continuous preset monitoring period refers to a time window in frames, the length of which is determined by the delay constraint of the audio system, and is commonly set to 10-50 frames to ensure that it is neither too sensitive to trigger frequently nor too long to miss critical mutations; the preset noise energy value refers to a fixed threshold set according to the human ear perceptible threshold and the system noise floor, typically between -70dBFS and -55dBFS.

[0126] In addition, it should be noted that the triggering mechanism adopts a two-stage strategy of "soft first and hard second": when the average noise energy value first exceeds the preset threshold, the system first uses soft optimization - only adjusts the loss function weight; if it is still not below the threshold for three consecutive monitoring periods, it enters hard optimization - starts grid search and temporarily freezes the audio output to avoid users hearing popping sounds.

[0127] In a feasible implementation, the step of triggering the model optimization of the stopband residual noise dynamic compensation model includes steps A41-A42: Step S41, extracting noise samples from the smooth transition signal, and updating the loss function weight of the stopband residual noise dynamic compensation model based on the noise samples; The noise samples refer to the residual high-frequency components separated from the smooth transition signal in the continuous monitoring period by spectral subtraction or mask filtering, and the time length is the same as the monitoring period; the loss function weight refers to the weighting coefficient in the MSE loss for different frequency bands or different time periods, the initial value is uniformly 1.0, and after updating, a higher weight (such as 5.0) is given to the stopband frequency range to enhance the sensitivity of the network to the error of this frequency range; the updating method adopts online gradient accumulation: the system calculates the error between each frame of noise samples and the corresponding target zero signal (in ideal case, the noise should be zero), and then performs a weight fine-tuning after accumulating a small batch (such as 32 frames), so as to balance stability and real-time performance.

[0128] In addition, it should be noted that, in order to avoid overfitting extreme samples in the mutation interval, the system also introduces a regularization term - applying L2 norm penalty to the weight increment, and the coefficient λ is 1e-4, so that the updated network can suppress noise without significantly changing the response to normal audio.

[0129] In a possible implementation, the noise sample extraction can employ an adaptive threshold spectral subtraction: first, perform STFT on the smooth transition signal to obtain an amplitude spectrum, then use the average spectrum of the adjacent noise-free frames as an estimated noise spectrum, and set to zero the time-frequency units lower than the estimated noise spectrum + 3 dB, and the remaining noise samples are retained.

[0130] Step A42, the hyperparameters of the stopband residual noise dynamic compensation model are adjusted by using a grid search.

[0131] The grid search in the embodiments of the present application refers to enumerating all combinations in a preset hyperparameter space, and selecting the optimal configuration by a validation set error index; the hyperparameters include but are not limited to GRU hidden layer dimension, learning rate, batch size, and dropout ratio. The validation set is composed of 10 pieces of different style audio collected historically, each piece containing at least one parameter mutation; the error index adopts a frequency band weighted signal-to-noise ratio.

[0132] In a possible implementation, the grid search can employ a hierarchical strategy: first, perform coarse-grained search on the hidden layer dimension and the learning rate, determine the optimal interval, and then perform fine-grained tuning on the dropout and the batch size, so as to reduce the calculation amount.

[0133] In the embodiments, the average noise energy value is calculated in real time by performing short-time Fourier transform on the smooth transition signal in the group band frequency band, and then the value is used as a criterion to trigger the model optimization of the stopband residual noise dynamic compensation model in a continuous preset monitoring period; after triggering, the noise samples are accurately extracted from the smooth transition signal, and the loss function weight is updated online, so that the sensitivity of the neural network to the stopband residual error is suddenly increased, and then the grid search is performed offline to re-lock the optimal hyperparameter combination. Thus, while maintaining the advantages of millisecond-level low delay and non-destructive editing of the digital system, the physical smoothing characteristics possessed by the analog filter due to the capacitor charging and discharging inertia can be continuously and adaptively approximated, the burst noise energy at the parameter switching instant can be suppressed, and real-time, non-mutated, reproducible high-fidelity audio filtering can be achieved.

[0134] In order to better help understand the scheme proposed in the present application, the following will combine Figure 5 , Figure 5 the overall flowchart of the stopband residual compensation method provided by the present application. As shown in Figure 5 , in the model training stage, a "hardware-digital dual-mode synchronous acquisition architecture" is built inside the digital audio workstation to simultaneously obtain the unprocessed original audio signal X (n) , the digital filter preliminary output Y 1(n) , and the output Y (n)and the original audio signal and the digital filter signal are fed as a dual-channel input to the deep recurrent network for training to generate a preliminary prediction signal Y 2(n) .

[0135] Further, the preliminary prediction signal and the digital filter signal are superimposed to obtain a final prediction signal , and a total loss value between the final prediction signal and the target label is calculated. Specifically, a mean square error is used as a time domain loss function, and a short-time Fourier transform is used as a frequency domain loss function. A time domain loss value between the final prediction signal and the target label is calculated according to the time domain loss function, and a fast Fourier transform is performed on the final prediction signal and the target label. A frequency domain loss value of the final prediction signal and the target label after the fast Fourier transform is calculated according to the frequency domain loss function. The total loss value is obtained according to the time domain loss value and the frequency domain loss value. Iterative training of the model is performed according to the total loss value, and a trained model is finally obtained.

[0136] In the model inference stage, when a user changes the cutoff frequency of the digital filter in real time through a MIDI CC (MIDI Continuous Controller), the digital filter immediately outputs Y 1(n) according to the new parameters, and this output is simultaneously input into the model together with the original signal X (n) after alignment at the sampling stage. The model processes the signals in an extremely low delay, and finally obtains a smooth transition signal Y 2(n) that maintains the advantages of flexible editing of a digital system and suppresses the residual energy in the stopband to below -70 dB, thereby achieving non-delay smooth transition and stopband residual suppression of the digital filter when the cutoff frequency is adjusted in real time at a high frequency.

[0137] The application also provides a stopband residual compensation device for a digital filter, which is described in detail in Figure 6 , and the stopband residual compensation device for the digital filter comprises: A multi-mode synchronous data acquisition module 10 is configured to obtain an original audio signal and a digital filter signal based on a pre-deployed hardware digital dual-mode synchronous architecture. A deep learning compensation filtering module 20 is configured to input the digital filter signal and the original audio signal into a pre-constructed stopband residual noise dynamic compensation model for processing to obtain a smooth transition signal, and the stopband residual noise dynamic compensation model takes a hardware filter signal output by a hardware analog filter as a target label. A superimposition module 30 is configured to superimpose the smooth transition signal and the digital filter signal at a preset superimposition ratio to obtain a calibrated signal.

[0138] The digital filter stopband residual compensation device provided by the present application adopts the digital filter stopband residual compensation method in the above embodiment, and can solve the technical problem of digital filter stopband residual compensation. Compared with the prior art, the digital filter stopband residual compensation device provided by the present application has the same beneficial effects as the digital filter stopband residual compensation method provided by the above embodiment, and other technical features of the digital filter stopband residual compensation device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0139] The present application provides a digital filter stopband residual compensation device, which comprises at least one processor and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the digital filter stopband residual compensation method in the above embodiment one.

[0140] Reference will be made to the following Figure 7 , which shows a structural schematic diagram of a digital filter stopband residual compensation device suitable for implementing the embodiments of the present application. The digital filter stopband residual compensation device in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, as well as fixed terminals such as digital TVs, desktop computers, and the like. Figure 7 The digital filter stopband residual compensation device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0141] As Figure 7As shown, the digital filter stopband residual compensation device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory 1002 or programs loaded from a storage device 1003 into a random access memory 1004. Various programs and data required for the digital filter stopband residual compensation device to operate are also stored in the random access memory 1004. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other by a bus 1005. An input / output interface 1006 is also connected to the bus. Generally, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the digital filter stopband residual compensation device to communicate with other devices wirelessly or by wire to exchange data. Although the digital filter stopband residual compensation device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0142] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0143] The digital filter stopband residual compensation device provided by the present disclosure adopts the digital filter stopband residual compensation method in the above embodiments, and can solve the technical problem of digital filter stopband residual compensation. Compared with the prior art, the digital filter stopband residual compensation device provided by the present disclosure has the same beneficial effects as the digital filter stopband residual compensation method provided by the above embodiments, and other technical features in the digital filter stopband residual compensation device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0144] It should be understood that various aspects of the disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the description of the embodiments above, specific features, structures, materials or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0145] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Any modifications or equivalents of the technology disclosed herein should be construed as falling within the scope of the application. Therefore, the scope of the application should be determined by the scope of the claims.

[0146] The application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., computer programs) for performing the stopband residual compensation method of the digital filter in the above-described embodiments.

[0147] The computer readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), etc., or any suitable combination of the above.

[0148] The above computer readable storage medium can be included in the stopband residual compensation device of the digital filter, or can exist separately without being assembled into the stopband residual compensation device of the digital filter.

[0149] The computer readable storage medium stores one or more programs, which when executed by the digital filter stopband residual compensation device, cause the digital filter stopband residual compensation device to: collect an original audio signal and a digital filter signal; input the digital filter signal and the original audio signal into a pre-constructed stopband residual noise dynamic compensation model for processing to obtain a smooth transition signal, the stopband residual noise dynamic compensation model taking a hardware filter signal output by a hardware analog filter as a target label; and superimpose the smooth transition signal and the digital filter signal at a preset superimposition ratio to obtain a calibration signal.

[0150] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0151] The flow and block diagrams in the drawings show architectural, functional, and operational representations of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow and block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or combinations of hardware and software.

[0152] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0153] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the above-mentioned stopband residual compensation method of the digital filter, and can solve the technical problem of stopband residual compensation of the digital filter. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the stopband residual compensation method of the digital filter provided by the above-mentioned embodiments, and will not be described here.

[0154] The present application also provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the above-mentioned stopband residual compensation method of the digital filter.

[0155] The computer program product provided by the present application can solve the technical problem of stopband residual compensation of the digital filter. Compared with the prior art, the computer program product provided by the present application has the same beneficial effects as the stopband residual compensation method of the digital filter provided by the above-mentioned embodiments, and will not be described here.

[0156] The above-mentioned is only some embodiments of the present application, and does not limit the patent scope of the present application, and any equivalent structural transformation, direct / indirect application in other related technical fields made by using the content of the present application specification and drawings under the technical concept of the present application are included in the patent protection scope of the present application.

Claims

1. A stopband residue compensation method for a digital filter, characterized by, The stopband residual compensation method of the digital filter comprises: Collecting an original audio signal and a digital filter signal; Synchronously inputting the digital filter signal and the original audio signal into a pre-constructed stopband residual noise dynamic compensation model for processing to obtain a smooth transition signal, the stopband residual noise dynamic compensation model taking a hardware filter signal output by a hardware analog filter as a target label; Superimposing the smooth transition signal and the digital filter signal at a preset superimposition ratio to obtain a calibration signal.

2. The stopband residual compensation method of claim 1, wherein, The step of synchronously inputting the digital filter signal and the original audio signal into the pre-constructed stopband residual noise dynamic compensation model for processing to obtain a smooth transition signal further comprises: Collecting a training data set based on a pre-deployed hardware digital dual-mode synchronous architecture, the data format of the training data set being pure time domain sampling data, the training data set comprising a hardware analog filter signal, an original audio training signal and a digital filter training signal; Taking the hardware analog filter signal as a target label, inputting the original audio training signal and the digital filter training signal into a deep recurrent network model to generate a preliminary prediction signal; Determining a final prediction signal according to the preliminary prediction signal and the digital filter training signal; Calculating a total loss value between the final prediction signal and the target label, updating parameters of the deep recurrent network model based on the total loss value, and iteratively performing the steps of taking the hardware analog filter signal as a target label, inputting the original audio training signal and the digital filter training signal into a deep recurrent network model to generate a preliminary prediction signal until the total loss value reaches loss convergence; Under the condition that the total loss value reaches loss convergence, taking the iterated deep recurrent network model as a stopband residual noise dynamic compensation model.

3. The stopband residual compensation method of claim 2, wherein, The step of collecting a training data set based on a pre-deployed hardware digital dual-mode synchronous architecture further comprises: Loading a digital filter plug-in and an original playback track, and forming a digital plug-in track based on the digital filter plug-in; Connecting a pre-deployed hardware analog filter through an audio interface loop to obtain a hardware processing track; Synchronizing cutoff frequency control parameters of the hardware processing track and the digital plug-in track, the cutoff frequency control parameters being obtained after the control signals are obtained based on automatic curve mapping; Building a hardware digital dual-mode synchronous architecture according to the hardware processing track, the digital plug-in track and the original playback track.

4. The stopband residual compensation method of claim 2, wherein, The step of calculating a total loss value between the final prediction signal and the target label comprises: Using a mean square error as a time domain loss function and a short-time Fourier transform as a frequency domain loss function; Calculating a time domain loss value between the final prediction signal and the target label according to the time domain loss function; Performing a fast Fourier transform on the final prediction signal and the target label, and calculating a frequency domain loss value of the final prediction signal and the target label after the fast Fourier transform according to the frequency domain loss function; Obtaining a total loss value according to the time domain loss value and the frequency domain loss value.

5. The stopband residual compensation method of claim 4, wherein, The step of calculating the time domain loss value between the final prediction signal and the target label according to the time domain loss function further comprises the steps of: acquiring physical characteristic parameters of a hardware analog filter in the hardware digital dual-mode synchronization architecture; constructing a prior constraint term based on the physical characteristic parameters, and integrating the prior constraint term into the time domain loss function.

6. The stopband residual compensation method of claim 1, wherein, The step of synchronously inputting the digital filtering signal and the original audio signal into the pre-constructed stopband residual noise dynamic compensation model for processing to obtain a smooth transition signal comprises the steps of: synchronously inputting the digital filtering signal and the original audio signal into the pre-constructed stopband residual noise dynamic compensation model at a sample point level for stopband residual noise dynamic compensation, and outputting a smooth transition signal point by point; The stopband residual noise dynamic compensation comprises the steps of: capturing features of the digital filtering signal and the original audio signal point by point by a multi-level gated recurrent unit of the stopband residual noise dynamic compensation model to obtain transient features; mapping the transient features to compensation coefficients by a fully connected layer of the stopband residual noise dynamic compensation model, and calculating dynamic compensation parameters based on the compensation coefficients.

7. The stopband residual compensation method of claim 1, wherein, The step of synchronously inputting the digital filtering signal and the original audio signal into the pre-constructed stopband residual noise dynamic compensation model for processing to obtain a smooth transition signal further comprises the steps of: monitoring a group band frequency range of the smooth transition signal, and calculating an average noise energy value in the group band frequency range based on a short-time Fourier transform; if the average noise energy value is higher than a preset noise energy value in a continuous preset monitoring period, triggering model optimization of the stopband residual noise dynamic compensation model, wherein the step of triggering model optimization of the stopband residual noise dynamic compensation model comprises the steps of: extracting noise samples from the smooth transition signal, and updating a loss function weight of the stopband residual noise dynamic compensation model based on the noise samples; adjusting hyperparameters of the stopband residual noise dynamic compensation model by using a grid search.

8. A stopband residue compensation device for a digital filter, characterized by, The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the stopband residual compensation method of the digital filter according to any one of claims 1 to 7.

9. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the stopband residual compensation method of the digital filter according to any one of claims 1 to 7.

10. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by a processor to implement the steps of the stopband residual compensation method of the digital filter according to any one of claims 1 to 7.

Images