A Real-Time Audio Effects Processing Method for Electronic Music

Abstract

This invention discloses a real-time audio effects processing method suitable for electronic music, relating to the field of audio signal processing. The method includes: real-time acquisition of an electronic music audio stream; continuous extraction of auditory feature vectors from the audio stream, consisting of logarithmically compressed subband energy coefficients and inter-frame nonlinear difference vectors; inputting the auditory feature vectors into a neural differential state evolver to update the hidden state at a content-adaptive rate and generate a modulation state trajectory. The parameters of this evolver are obtained through self-supervised training, which utilizes the inter-frame spectral similarity of the audio to construct a contrast loss and updates the parameters through gradient backpropagation; mapping the modulation state trajectory to control parameters of a digital signal processing effects unit; and using the control parameters to perform real-time effects processing on the audio stream and outputting the results. This invention achieves natural synchronization between dynamic effect parameters and music segment transitions under conditions of no beat detection, timecode, or manual / automatic operation.

Description

Technical Field

[0001] This invention relates to the field of audio signal processing, and more specifically, to a real-time audio effects processing method suitable for electronic music. Background Technology

[0002] In live electronic music performances and real-time audio processing applications, shaping the dynamic evolution of sound by continuously changing effects parameters such as filter cutoff frequency and reverberation time is a common technique. Currently, the main solutions for achieving automatic parameter modulation fall into the following categories.

[0003] One type is the signal envelope-based tracking method, which uses a low-frequency oscillator or envelope follower to directly map changes in audio amplitude to parameter values. This method can only reflect the physical loudness of the signal and cannot perceive the segmentation and structural changes within the music. Another type is the synchronization method based on beat information. For example, Chinese Patent No. CN118942481A discloses an audio processing method that requires extracting parameters such as beat list, beat type, and beat speed from the audio file. The switching of parameters depends on the explicit beat position, and the resulting modulation trajectory is often stepped or periodically repetitive, making it difficult to generate continuous changes that naturally match the flow of music. A third type is the switching method based on preset time points. For example, Chinese Patent No. CN112133267B discloses an audio effect processing method, device, and storage medium. This method obtains the parameters before and after the switch by determining the pre-calibrated effect parameter switching time points in the target audio and performs transition processing. This method requires pre-arranged time points and cannot automatically adjust according to real-time audio content.

[0004] In summary, existing solutions, when implementing automatic parameter modulation, either rely solely on amplitude envelope or require external information such as beat and preset time points. Neither can capture the timing of parameter changes from the content structure of the audio signal itself, resulting in a lack of coordination between the dynamic evolution of the effect and the natural direction of the music.

[0005] Therefore, a real-time audio effects processing method suitable for electronic music is proposed to address the above problems. Summary of the Invention

[0006] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present invention provide a real-time audio effects processing method suitable for electronic music. It aims to solve the problem that the automatic modulation of effects parameters in the prior art depends on beat detection, preset time points or amplitude envelopes, and it is difficult to automatically generate a continuous parameter evolution trajectory that is naturally synchronized with the transition of music segments based on the structural changes of the audio content itself.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A real-time audio effects processing method suitable for electronic music includes steps S1 to S5.

[0009] S1. Real-time acquisition of electronic music audio streams as the input signal source for subsequent feature extraction and effects processing.

[0010] S2. Continuously extract auditory feature vectors from the audio stream, wherein the auditory feature vectors are composed of log-compressed subband energy coefficients and inter-frame nonlinear difference vectors.

[0011] The subband energy coefficient reflects the spectral energy distribution of the current frame, while the inter-frame nonlinear difference vector reflects the changing trend of the spectral structure between adjacent frames. The combination of these two features makes the extracted features sensitive to transient shocks and changes in spectral profile, providing an input basis for the subsequent state evolver to perceive changes in music structure.

[0012] The inter-frame nonlinear difference vector is calculated using the logarithmic ratio of the energy coefficients of corresponding sub-bands in adjacent frames. This logarithmic ratio format amplifies transient changes in weak-energy sub-bands and prevents high-energy sub-bands from dominating the difference calculation results.

[0013] By performing a weighted average of the difference vectors of several consecutive frames, short-term texture trend features are obtained, which preserve the directional information of spectral evolution while smoothing instantaneous fluctuations.

[0014] S3. Input the auditory feature vector into the neural differential state evolver, which updates the hidden state at a content-adaptive rate and generates a modulated state trajectory.

[0015] Through a content-adaptive rate mechanism, the state update speed can be automatically adjusted to follow the changing rhythm of the audio content:

[0016] When the audio changes drastically, the state is updated with finer steps to track rapid transients in a timely manner.

[0017] When the audio is stable, the state update step size is increased or only a single step is updated to keep the parameters stable.

[0018] The specific method for content-adaptive rate update of hidden state is as follows: within each frame interval, a transient change rate index is calculated based on the current auditory feature vector. The higher the index, the more integral substeps are required for state update.

[0019] Furthermore, the transient rate of change index is calculated by using the sigmoid function to obtain the inner product of the learnable vector and the auditory feature vector. The learnable vector is automatically adjusted during training, enabling the network to learn which feature dimensions are more important for judging transient changes.

[0020] The parameters of the neural differential state evolver are obtained through self-supervised training. Self-supervised training constructs a contrast loss using the inter-frame spectral similarity of the audio and updates the parameters through gradient backpropagation, enabling the state evolver to gradually learn differentiated state change behaviors between the transition regions and stable regions of the musical structure, without requiring any manually labeled structural boundary information.

[0021] The specific process of self-supervised training includes: calculating the slow average feature vector and the fast average feature vector, where the time constant of the slow average is greater than that of the fast average.

[0022] Slow averaging reflects a stable spectral texture over a longer period, representing the baseline state within a segment;

[0023] Fast averages reflect instantaneous changes over shorter periods and are more sensitive to musical transitions.

[0024] A binary mask is generated based on the difference between the slow-average feature vector and the fast-average feature vector to identify structural transition regions and structurally stable regions. When the difference between the fast and slow averages exceeds an adaptive threshold, it indicates that the spectral texture of the current frame has deviated significantly from the recent stable state, and is thus identified as a structural transition region.

[0025] Using the above method, the structural boundaries of musical segments are automatically detected without relying on manual annotation.

[0026] Furthermore, the slow average eigenvector and the fast average eigenvector are calculated using an exponential moving average, with the smoothing coefficient determined by the time constant and the frame interval. The exponential moving average assigns higher weights to more recent frames and gradually reduces the weights of more distant frames, thus smoothing noise while maintaining a fast response to recent changes.

[0027] The contrast loss is constructed as follows:

[0028] Calculate the inter-frame spectral similarity matrix of the audio to reflect the degree of spectral similarity between the frames of the processed audio.

[0029] The binary mask is used to divide the frame pairs into transition frame pairs and stable frame pairs;

[0030] The loss is constructed using the average dissimilarity of frame pairs in the transition region and the average dissimilarity of frame pairs in the stable region. The dissimilarity is calculated from the spectral similarity.

[0031] Minimizing this loss will force the spectral difference between frame pairs in the transition region to increase and the spectral difference between frame pairs in the stable region to decrease, thereby driving the state evolver to increase the amplitude of state changes in the structural transition region and suppress the amplitude of changes in the stable region.

[0032] Furthermore, the average dissimilarity is only statistically analyzed for frame pairs whose time intervals are within a preset window. Window limitation focuses the training signal on structural coherence changes within a local time range, avoiding irrelevant spectral differences introduced by long-distance frame pairs and improving the accuracy of the training signal.

[0033] S4. Map the modulation state trajectory to the control parameters of the digital signal processing effects unit to realize the conversion from the hidden state to the specific effects unit parameter values.

[0034] The mapping method is as follows: an intermediate quantity is obtained through linear mapping, and then the intermediate quantity is mapped to the effective range of each parameter through nonlinear transformation, so that any value of the hidden state can correspond to the actual value range of the effector parameter.

[0035] S5. The audio stream is processed in real time using the control parameters to output a modulated audio signal. The effect parameters change continuously with the state trajectory, ensuring that the dynamics of effects such as filtering and reverberation are synchronized with the structural changes in the audio content.

[0036] During the self-supervised training phase, the digital signal processing effector is differentiable, and the gradient of the comparison loss is backpropagated to the state evolver via the effector, achieving end-to-end parameter updates. During the real-time inference phase, the digital signal processing effector is replaced with a non-differentiable implementation, reducing the computational overhead of real-time processing while maintaining the modulation capability obtained during training.

[0037] The technical effects and advantages of this invention are as follows:

[0038] In this method, the training of the neural differential state evolver requires no manual annotation. During training, by comparing feature changes at different time scales of audio segments, the system automatically identifies structural transitional and stable segments of the music. Training objectives are constructed using the inter-frame spectral similarity of the processed audio, enabling the evolver to learn to proactively adjust its state at segment transitions and maintain a stable state within segments. In practical use, the evolver automatically adjusts its state update speed based on the intensity of changes in the current audio, responding quickly to rapid changes and remaining stable when changes are gradual, generating a smooth parameter trajectory. This trajectory is then converted into control parameters for effects such as filters and reverb, allowing the effect parameters to automatically change with the fluctuations of the music segments.

[0039] Since the entire processing only uses the content structure of the input audio itself as the driving signal, it does not require beat detection, preset time points, or manual drawing of parameter curves. This method helps to make the dynamics of the effect and the musical structure more natural, providing a coherent and layered real-time audio effects processing solution for live electronic music performances. Attached Figure Description

[0040] Figure 1This is an overall flowchart of the real-time audio effects processing method of the present invention;

[0041] Figure 2 This is a schematic diagram of the self-supervised training process of the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Example 1

[0044] As attached Figures 1 to 2 The method described is a real-time audio effects processing method suitable for electronic music. This method enables the neural differential state evolver to automatically adjust the evolution rate and amplitude of the hidden state according to the structure of the audio content through self-supervised training. In real-time inference, it generates effect parameter modulation trajectories that are naturally synchronized with the transition of music segments. It can bring coherent and structured effect dynamics to live electronic music performances without the need for beat detection, timecode, or manual parameter automation.

[0045] The implementation process of this method will be explained in detail below, in conjunction with steps S1 to S5.

[0046] S1. Real-time acquisition of electronic music audio streams

[0047] First, step S1 is executed to acquire the electronic music audio stream in real time. The source of the audio stream determines the input format and buffer configuration for subsequent processing.

[0048] In one implementation, the audio stream originates from the real-time track output of a digital audio workstation. The audio engine populates the current track's audio data into the processing buffer as a plug-in, with the sampling rate fs set to 44100Hz, sending in 128 samples at a time. The processing thread uses this buffer block as the smallest processing unit.

[0049] As another implementation, the audio stream comes from the hardware synthesizer via real-time input through an audio interface. In this case, fs is 48000Hz, and the number of samples sent each time is determined by the audio interface driver, for example, 256 samples.

[0050] In another implementation, the audio stream originates from the amplified sound signal of an electronic musical instrument picked up by a live microphone. The signal is pre-amplified and converted from analog to digital before being sent to the processing thread.

[0051] Regardless of the source, the audio stream is continuously fed in at a fixed sampling rate. The processing thread maintains a sliding window buffer for overlapping frame splitting in subsequent steps. If the input audio is stereo, it is first mixed into mono for feature extraction, and then the same control parameters are applied to the left and right channels respectively during final effects processing to maintain spatial positioning unaffected by modulation.

[0052] S2. Continuous extraction of auditory feature vectors

[0053] After acquiring the audio stream, step S2 is executed to continuously extract auditory feature vectors c from the audio stream. These auditory feature vectors are composed of two concatenated parts:

[0054] One part is the subband energy coefficient of logarithmically compressed, which reflects the energy distribution of each frequency band in the current frame;

[0055] The other part is the short-term texture trend feature obtained by weighted averaging of the inter-frame nonlinear difference vectors, which reflects the evolution trend of the spectral structure over time.

[0056] As one implementation method, a gamma-pass filter bank is used to extract the sub-band energy coefficients. The filter bank contains G channels, where G is 24, and the center frequencies of each channel are distributed between 50Hz and 8000Hz according to the equivalent rectangular bandwidth scale. For each frame of audio signal, it is passed through this filter bank, and the root mean square value of the output signal of each channel is calculated as the energy value of that sub-band. The 24 energy values ​​are then compressed by taking the natural logarithm of each value to form a 24-dimensional vector e.

[0057] Frame shift H is taken as 128 sampling points, frame interval .

[0058] make Indicates e in the current frame. The i-th element of the inter-frame nonlinear difference vector d represents e from the previous frame. ,in Pick The difference vectors of three consecutive frames are weighted and averaged with weights [0.5, 0.3, 0.2] to obtain short-term texture trend features. . e and The vectors are spliced ​​together to form a 48-dimensional auditory feature vector c.

[0059] As another implementation, the difference vectors of five consecutive frames can be taken and weighted by exponential decay, for example, with weights of 0.4, 0.25, 0.15, 0.1, and 0.1 respectively, to introduce texture change information over a longer time range.

[0060] As another implementation, the subband energy coefficient is extracted using a constant Q transform. A constant Q transform is performed on each frame of the signal, with the frequency axis divided on a logarithmic scale. Twelve subbands are taken per octave, with a minimum center frequency of 32.7 Hz and a maximum of 16744 Hz, for a total of 24 subbands. The subband energy is obtained by summing the squares of the amplitude spectrum within each subband, and then compressed using the natural logarithm to obtain e.

[0061] The inter-frame nonlinear difference vector d is expressed in the form of a difference ratio:

[0062] ;

[0063] The difference vectors of five consecutive frames are obtained by weighting them with exponential decay. , with e The result is c by splicing.

[0064] As another implementation method It is obtained by performing a first-order low-pass filter on the difference vector sequence. The time constant corresponding to the smoothing coefficient is 3 times the frame interval, making... Smoothly track the changing trend of the difference vector.

[0065] S3. The neural differential state evolver generates modulated state trajectories.

[0066] Next, step S3 is executed, inputting the auditory feature vector c into the neural differential state evolver. This evolver updates the hidden state h at a content-adaptive rate, generating a continuous and smooth modulated state trajectory, the parameters of which are obtained through self-supervised training. The state evolution process in real-time inference and the offline self-supervised training process are described below.

[0067] The implementation of state evolution in real-time reasoning is as follows:

[0068] During the real-time inference phase, the evolver maintains a hidden state h, which is a 16-dimensional vector initialized to zero. The temporal evolution of the hidden state is governed by neural differential equations. Control, among which It is a fully connected neural network.

[0069] As one implementation method, The structure is as follows:

[0070] The first fully connected layer maps the concatenated vector of h and c (64 dimensions in total) to 32 dimensions, and then activates it with tanh.

[0071] The second fully connected layer maps 32 dimensions to 16 dimensions, and outputs the state change rate after tanh activation. The tanh activation function will Each component is naturally limited to Within the specified range, the numerical values ​​are guaranteed to be stable.

[0072] Within each frame interval, the evolver updates its state according to the following procedure. First, a learnable vector v (48-dimensional) and a learnable scalar b are introduced, and a transient rate of change index is calculated. . Between 0 and 1 The larger the value, the denser the transients in the current frame and the more drastic the music changes.

[0073] Then, according to Determine the number of integral substeps within this frame. K is the preset maximum number of extra substeps:

[0074] when When N is close to 1, the state is relatively large, and the state is updated in more detailed substeps to track rapid changes;

[0075] when Approaching 0 The state is updated only in a single step to keep the parameters stable.

[0076] The value of K can be flexibly adjusted according to the target platform: 2 on embedded processors to reduce computational overhead, and 6 on high-performance computers to obtain higher dynamic response accuracy.

[0077] Finally, perform integral propagation. Adjust the frame interval... Divide into N equal substeps, each substep having a length of Starting from the current state h, execute each sub-step sequentially:

[0078] , ;

[0079] when When that happens, the Euler method is used directly. After N iterations, the end state of the current frame is obtained, which serves as the starting state for the next frame. Connecting the state values ​​frame by frame forms a continuous and smooth modulation state trajectory.

[0080] As another implementation method Using residual structure: ,in It is a two-layer fully connected network, with the structure described above. Similarly, W is learnable. Linear projection matrix. This structure allows the network to directly learn the rate of change of the state relative to the increment of the current state, which helps with gradient propagation and training stability.

[0081] As another implementation method, the integration solution uses the fourth-order Runge-Kutta method to obtain higher accuracy.

[0082] As another implementation, the activation function replaces tanh with a scaled sigmoid, limiting the rate of change to a preset range. Inside, This is the scaling factor.

[0083] The implementation of offline self-supervised training is as follows:

[0084] The key to the structure-aware capability of the state evolver lies in offline self-supervised training. The training data consists of a large number of dry electronic music clips, each 4 to 8 seconds long, without any effects processing or structural boundary annotations.

[0085] For each training segment, extract the feature vector sequence according to step S2. Training first generates a binary mask sequence M, indicating whether each frame is in a structural transition region or a structurally stable region.

[0086] Specifically, both the slow average eigenvector s and the fast average eigenvector f are maintained simultaneously.

[0087] (1) Slow averaging with time constant Perform exponential moving average, smoothing coefficient The updated formula is The initial value is taken as It reflects a stable spectral texture over a longer period of time.

[0088] (2) Fast averaging with time constant Perform exponential moving average, smoothing coefficient The updated formula is The initial value is taken as It reflects instantaneous changes over a relatively short period of time.

[0089] Next, the difference metric for each frame is calculated. This refers to the Euclidean distance between the fast average and the slow average. An adaptive threshold is defined. ,in L frames prior to the current frame The average value, Standard deviation Take 2. The value of L is related to the slow averaging time constant. Related, when Taking L as 100 covers a time span of approximately 290ms, which is sufficient to establish a stable statistical benchmark.

[0090] when At that time, the spectral texture of the frame deviates significantly from the recent stable state, and is marked as a structural transition region, indicated by a binary mask. Otherwise, it is marked as a structurally stable region. This generates an M of the same length as the feature sequence.

[0091] As an alternative, slow and fast averages can be replaced with equal-weighted sliding window averages, with window lengths set to... and And round down. The difference measure can also use the L1 norm, and the threshold can be changed to the first 200 frames. The 90th percentile.

[0092] After generating M, a contrastive loss training state evolver is constructed. The training segment itself is used as input, and the state trajectory is generated by the state evolver under the current parameters. Then, it is converted into control parameters through the mapping described in step S4, which drives the differentiable effector to process the segment, resulting in the processed audio.

[0093] The processed audio is divided into frames using the same frame parameters as S2. The amplitude spectrum is extracted and L2 normalized. The cosine similarity matrix S between the frames is calculated. S[i,j] is the cosine similarity between the normalized amplitude spectrum vectors of the i-th frame and the j-th frame, with a value range of [0,1]. The closer the value is to 1, the more similar the spectra of the two frames are.

[0094] Frame pairs are divided using M:

[0095] Transition frame pairs satisfy and Stable frame pairs satisfy and Calculate the average dissimilarity of frame pairs in the transition region. That is, the corresponding frame pair The average value;

[0096] Calculate the average dissimilarity of frame pairs in the stable region. That is, the corresponding frame pair The average value.

[0097] Only those conditions are considered in the statistics. The neighboring frame pairs, W is set to 30. The window W determines the time scale of structural changes of interest: at a frame interval of approximately 2.9ms. With a coverage of approximately 87ms, it is suitable for capturing common passage transitions and rhythmic changes in electronic music; if processing slower music genres, W can be increased appropriately, for example, 50 corresponds to approximately 145ms.

[0098] Comparative loss Minimizing this loss forces Enlarge This reduces the amplitude of state changes in the structural transition region and suppresses them in the stable region by driving the state evolution engine. Alternatively, contrastive loss can also be used. .

[0099] Furthermore, an entropy regularization term is introduced to encourage independent responses from different structural events to each dimension of the hidden state. The variance of each dimension of the hidden state in all frames within the training batch is calculated and normalized to obtain the distribution p, and the entropy regularization term is then applied. Total loss , Set the value to 0.1. Use the Adam optimizer with a learning rate of 0.1. The batch size is 32 segments, and the iteration continues until the loss converges.

[0100] Save all parameters, including the network parameters, after training is complete. The weights, vector v, and scalar b, and the mapping matrix U and bias from step S4. It can be used directly in real-time inference.

[0101] S4. Map to effects control parameters

[0102] After the state trajectory is generated, step S4 is executed to map the modulation state trajectory to the control parameters of the digital signal processing effects unit.

[0103] As one implementation, suppose the effect has M adjustable parameters. Introduce a linear mapping matrix U( and bias vector (M-dimensional), calculate intermediate quantities Let the valid range of the j-th parameter be... ,pass Will Mapped to this range, where .

[0104] For example, when controlling the cutoff frequency of the filter Hz, Hz; when controlling the resonant gain dB dB; when controlling reverberation time s、 The specific upper and lower bounds of s can be adjusted according to the design of the reverb used. The mapping function can also be replaced by sigmoid or tanh.

[0105] As another implementation, the 16-dimensional hidden state is divided into multiple sub-states. Each sub-state is independently linearly mapped to generate one or a set of associated parameters, enhancing the difference and independence of the dynamic trajectories of each parameter.

[0106] S5. Real-time Effects Processing and Output

[0107] Finally, step S5 is executed, which uses control parameters to perform real-time effects processing on the audio stream and outputs the modulated audio. During the training phase, the effects are implemented using a differentiable method to enable gradient backpropagation; during the inference phase, a non-differentiable implementation is used to reduce computational overhead. This is achieved using the state evolver, mapping matrix U, and bias. It remains unchanged.

[0108] As one implementation, the effects unit employs a standard dual second-order filter. This filter is designed based on angular frequency. And the quality factor Q.

[0109] First calculate the intermediate quantity And obtain the unnormalized denominator coefficients: , , ,

[0110] and numerator coefficient , , .

[0111] Then, the coefficients are normalized to obtain the coefficients used in the difference equation:

[0112] , , , , .

[0113] This leads to the difference equation .

[0114] During the training phase, the trigonometric functions and division operations in the above coefficient calculations are implemented using differentiable methods, and the gradient can be backpropagated through these operations. And Q. During the inference phase, the coefficients are calculated using non-differentiable functions from the standard mathematical library and stored in registers. When the difference equation is executed, the coefficients are read directly without constructing a gradient graph. After processing, the audio is output to the audio interface playback buffer.

[0115] As another implementation, the effect uses a reverb unit constructed from a cascaded full-pass filter. The delay length D of the full-pass filter is typically a non-integer sample point, and the output value is synthesized from two adjacent integer sample points through linear interpolation during the training phase. For example, the delay line readout position is... When, the output is taken and The interpolation is linearly weighted. This interpolation operation is differentiable, and the gradient can be distributed to the two sampling points according to the weights. The feedback coefficients and delay length are driven by control parameters. During the inference phase, linear interpolation is replaced with nearest-neighbor sampling or fixed-coefficient interpolation to reduce computational cost.

[0116] In another implementation, the effects unit is a cascaded filter and reverb unit. The first two components of the control parameters drive the filter, and the last two components drive the reverb unit. Both modules are differentiable during the training phase, but are replaced with non-differentiable, high-efficiency implementations during the inference phase.

[0117] During the self-supervised training process in step S3, the differentiable effector described in step S5 is used to generate the processed audio and calculate the loss. After training convergence, the state evolver network... Mapping vector v and scalar b, linear mapping matrix U and bias The parameters are determined. During the real-time inference phase, these parameters are loaded and remain unchanged, with only the state evolution in S3, parameter mapping in S4, and forward processing of the non-differentiable effector in S5 performed, without any online optimization or parameter updates.

[0118] The entire processing chain is automated during inference without relying on beat detection, timecode, or manual parameters, and latency can be controlled within 5ms. Since the dynamic changes of effect parameters in each processing step are uniquely determined by the instantaneous content structure of the input audio, this method provides a coherent and structured dynamic evolution of effects for live electronic music performances.

[0119] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A real-time audio effects processing method suitable for electronic music, characterized in that, Including the following steps: S1. Real-time acquisition of electronic music audio streams; S2. Continuously extract auditory feature vectors from the audio stream, wherein the auditory feature vectors are formed by concatenating log-compressed subband energy coefficients and inter-frame nonlinear difference vectors; S3. Input the auditory feature vector into the neural differential state evolver, which updates the hidden state at a content-adaptive rate and generates a modulated state trajectory. The parameters of the neural differential state evolver are obtained through self-supervised training, which constructs a contrast loss using the inter-frame spectral similarity of audio and updates the parameters through gradient backpropagation. S4. Map the modulation state trajectory to control parameters of a digital signal processing effects unit; S5. Perform real-time effects processing on the audio stream using the control parameters, and output the modulated audio signal.

2. The real-time audio effects processing method for electronic music according to claim 1, characterized in that, The inter-frame nonlinear difference vector mentioned in step S2 is calculated by the logarithmic ratio of the energy coefficients of the corresponding sub-bands of adjacent frames; The short-term texture trend features are obtained by weighted averaging of the difference vectors of several consecutive frames.

3. The real-time audio effects processing method for electronic music according to claim 1, characterized in that, The method of content adaptive rate update of hidden state in step S3 is as follows: in each frame interval, a transient change rate index is calculated based on the current auditory feature vector. The higher the transient change rate index, the more integral substeps are required for state update.

4. The real-time audio effects processing method for electronic music according to claim 3, characterized in that, The transient rate of change index is calculated by using the sigmoid function to obtain the inner product of the learnable vector and the auditory feature vector.

5. The real-time audio effects processing method for electronic music according to claim 1, characterized in that, The self-supervised training includes: Calculate the slow average eigenvector and the fast average eigenvector, where the time constant of the slow average is greater than that of the fast average. A binary mask is generated based on the difference between the slow average eigenvector and the fast average eigenvector. The binary mask identifies the structural transition region and the structural stable region.

6. The real-time audio effects processing method for electronic music according to claim 5, characterized in that, The slow average feature vector and the fast average feature vector are calculated by exponential moving average, and the smoothing coefficient of the exponential moving average is determined by the time constant and the frame interval.

7. The real-time audio effects processing method for electronic music according to claim 5, characterized in that, The contrast loss is constructed as follows: Calculate the inter-frame spectral similarity matrix of the audio; The binary mask is used to divide the frame pairs into transition frame pairs and stable frame pairs; The loss is constructed using the average dissimilarity of transition region frame pairs and the average dissimilarity of stable region frame pairs, where the dissimilarity is calculated from spectral similarity.

8. The real-time audio effects processing method for electronic music according to claim 7, characterized in that, The average dissimilarity is only calculated for frame pairs whose time intervals are within a preset window.

9. The real-time audio effects processing method for electronic music according to claim 1, characterized in that, During the self-supervised training phase, the digital signal processing effector used in step S5 is differentiable; during the real-time inference phase, the digital signal processing effector is replaced with a non-differentiable one.

10. The real-time audio effects processing method for electronic music according to claim 1, characterized in that, In step S4, the method for mapping the modulation state trajectory to control parameters is as follows: an intermediate quantity is obtained through linear mapping, and then the intermediate quantity is mapped to the effective range of each parameter through nonlinear transformation.

Images